Abstract
In model-based virtual conference system, the facial expressions on human faces are major focus of all users. Many facial expression analysis algorithms for frontal face have been proposed. For practical use, we develop a facial expression analysis method that can allow users to feel free to rotate their heads in communication. Furthermore, in order to aid our facial expression analysis method, a pose refinement algorithm is proposed by using error classification method.In our expression analysis method, we translate all facial images under different head pose into artificial frontal facial images called stabilized view, and track facial feature points in these frontal facial images. We obtain initial locations of the feature points by the assistance of user-customized 3D facial model, and then track their movement by some information, such as shape, intensity, temporal correlation, lip color and lip texture. When head rotation is too large such that some feature points are hidden from view, we adopt symmetric assumption to estimate the locations of these hidden feature points. After acquiring the feature-point tracking result, we translate them into Facial Animation Parameters that control the animation of the talking head at the client terminal in virtual conference.In order to analyze facial expression under different head pose, we need to know exact pose information. In this thesis, we also propose a pose refinement algorithm that can refine head pose rapidly after coarse pose estimation. We adopt Fisherface classification method to classify pose error in a 2D difference image, and two classification schemes both using Fisherface method are designed in our system. First one, pose verification, is used to verify whether the estimated head pose is correct or not. If the pose is not correct, the other one, error type classification, is applied to determine what kind of pose error occurs, and then correct the error pose. The pose verification and error type classification are iteratively applied on difference image until the correct head gesture is obtained.