Abstract
To improve the mutual relationship between humans and machines, an emotion recognition system based on physiological signals is proposed. To study and verify this system, we collect a new emotional database, the National Taiwan University of Art database (NTUA database), and use performing induction method to spontaneously elicit subjects' emotional states. The database records subjects' performance video, audio and physiological signals (electrocardiogram and respiration). In this work, we focus on the physiological signals, and try to make it a reliable channel of emotion recognition. To recognize the subjects' emotion efficiently, we analyze the emotional labels and make use of them to build a two-dimensional valence-arousal model. In addition, not only the prevailing functional features from heart rate variability (HRV) in time domain and frequency domain, but also the features encoded by the Gaussian mixture mode (GMM) and Fisher vector are introduced to obtain new features. The latter is a common method in audio and video signal processing, but it is rarely conducted in physiological area. However, this method can truly reflect the series of physiological reaction, and improve the recognition accuracy. The support vector machine is proposed as the classifier to classify different levels of valence and arousal, and minimum redundancy maximum relevance (mRMR) is used as the strategy of feature selection to pick up the useful features. We analyzed the recognition results from each experiment step to prove that each process was rational, and compared the recognition accuracy between the features from HRV and the features from Fisher vector. The final accuracy of classification is 54.6% for valence and 61.1% for arousal. We also conduct the feature-level fusion of physiological and audio signals, and the recognition accuracy increases to 64.24% for valence and 63.31% for arousal.