Abstract
In virtual conferencing systems, the changes of facial objects on human faces are major focus of all users. In order to represent the detail changes of facial features, the algorithms for feature point tracking are developed to track motion of these features and expression analysis is applied to translate the tracking result to facial animation parameters for user’s understanding. Hence, in this thesis, we propose methods and algorithms for these two major problems. First, a real-time feature point tracking algorithm is presented to achieve medium to high accuracy with minimized computation complexity. The tracking algorithm is separated into two phases including the registration phase and the tracking phase. Initial locations of all feature points are obtained in registration phase and also some information is registered for the next phase. In the tracking phase, major feature points are tracked by spatial and temporal information. Taking advantage of spatial information such as contrast, mask value is used to describe characteristic of each feature point and temporal block template matching is also added for tracking temporal relations. In this algorithm, six salient feature points for eyes, four for eyebrows and six for mouth region are tracked. Although the real-time feature point tracking algorithm can track feature points on outer contour robustly, those inside the mouth cannot be tracked by ordinary methods such as contrast or block matching. A mesh-based method is presented here for solving the task inside the mouth. Comparing to the traditional mesh structure, a hierarchical mesh model is applied in our system. It can minimize the search space of the mesh model and have better performance. Similar to the real-time feature point tracking algorithm, mesh-based tracking method is divided in two phases. The registration phase performs the model adaptation for fitting the original mouth model to current user’s mouth region. Then in the tracking phase, all vertices in our three-leveled mesh model are adjusted in turns from coarse level to fine level by texture and color information. After the adjustment, the new shape of the mouth and motion of all vertices can be observed. Acquiring only the tracking result cannot be understood by users, and needs to be translated to the facial animation parameters. Since the facial animation parameters are dependent regionally, all relative parameters are supposed to be adjusted at the same time. Hence, the mapping becomes a many-to-many problem, and a modified least-squares algorithm is adopted to solve this mapping problem with minimized errors. After translating the tracking result to the facial animation parameters, low-level expression analysis is accomplished. From our experimental result, the real-time feature point tracking algorithm achieves required performance in tracking four dominant feature points on outer contours. The mesh-based method can solve the complexity situations inside the mouth given correct positions of four major control vertices on outer contours. From the tracking result of the previous two methods, the low-level expression analysis successfully interprets the motion of salient feature points to facial animation parameters with very low mapping errors.