Abstract
Regarding speech emotion recognition, so far the most commonly seen method is to use a human beings exterior behavioral signals (such as voice, images, words and body language) to build the machine learning model. Other research has tried to understand the process of emotion recognition via human beings’ interior physical signals (through electroencephalography, electrocardiography and functional magnetic resonance imaging). This study observes subjects while they listen to emotional speech, using magnetic resonance imaging (MRI) technology to build the recognition system for activation and valence. However, after the subjects were tested for a long time, there were some changes in their brain activities, such as tiredness and distraction; these extra brain activities were similar to noise, which influenced the effects of activation and valence. Therefore, we applied a joint factor analysis algorithm to the brain magnetic resonance imaging as a method of separating speech emotion recognition in the brain from temporal effect signals. Speech activation and valence effects in the results after the method was applied are significantly improved.