Abstract
Sparse coding technique has been proved to be very effective in extracting global features from signals for several different applications. Furthermore, the sparse representation was designed to produce sparse solution at the group level by considering group structure of training images. However, distinctive objects or different action videos usually contain multiple data attributes which are high-level descriptions about the properties of objects or actions. For the action recognition problem, action video may contain multiple attributes, such as different types of viewing angle, pose and illumination. Such multi-attribute properties cannot be fully exploited by the group lasso method since it is not designed to handle multiple attributes. In this thesis, we propose multi-attribute sparse representation based method enforced with group constraint for the action recognition and facial expression recognition problems which contain multiple data attributes. For the action recognition problem, an over-segmentation based background modeling and foreground detection approach is employed to extract silhouettes from action videos firstly. Then, multiple time intervals of the motion history images are computed to capture motion and pose information in human activities. Actions with multiple attributes can be represented by individual attribute matrices to describe group property for each action instance. These attribute matrices are incorporated into the formulation of l_1-minimization. The sparsity property as well as the group constraints makes the basis selection in sparse coding more efficient in term of accuracy. Especially, our approach is able to operate under the condition of partially labeled attributes in the training data. Furthermore, we integrate action units (AUs) information and multi-attribute sparse coding for facial expression recognition. AUs not only can be represented by an individual attribute mask to describe group property for each facial expression video, but also as a constraint to enforce that the same facial expressions should have very similar AUs. The group constraint makes the basis selection in sparse coding more efficient and the AU similarity constraint penalizes selecting the dictionary atoms with distance far away the target instance. These groups constraint and the AU similarity constraint are incorporated into the formulation of l_1-minimization to recognize facial expression. We will demonstrate the proposed multi-attribute sparse coding based method through experiments on several public multi-view human action datasets and facial expression datasets to show the effectiveness and robustness of the proposed method.