Real-time face tracking for video stream
The core of Glazfit products is 3D face tracking, a cutting-edge technology which can detect and track facial points (or keypoints) and estimate 6 Degrees-of-Freedom (DoF) including three dimensional movements and three dimensional directions of the faces in images or video frames. Different from other face tracking solutions, Glazfit technology targets Web-based applications and being optimized for virtual glasses try-on.
68 landmarks for an image in 300-W dataset (source: ibug)
3D face modeling: 51 facial points in synthetic images and its rendering at different orientations.
Glazfit detection/tracking algorithms rely on a large face dataset (about 10K images). From some public datasets, which are available in computer vision community, we use 3D face modeling algorithms to synthesize one 2D image in order to build a 3D face model, so that we can render it at different orientations (Yaw, Pitch and Roll) and create a much larger dataset.
We use some machine learning (regression based learning) and computer vision (local feature representation) techniques to learn detection/tracking models from such a large dataset in order to avoid the overfitting problem and allows the tracking system to be consistent with face orientation. Kalman Filtering (KF) algorithm is also used to smoothen the trajectory of facial movements and stablize the tracking system.