This paper presents a systematic optimization strategy to implement the complicated EBGM face recognition on low processing power mobile devices. We propose a tailor-made fixed point arithmetic and various memory access optimization techniques to speed up an EBGM authentication process from ~550s down to ~1s in a typical Intel PXA255 400 MHz mobile platform. The result shows that real time face recognition can be completed without any noticeable accuracy loss. This finding not only provides a guideline for porting various biometric applications in the mobile systems, but also exploits new opportunities for different mobile e-commerce applications with biometric identity checking.