Logo image
Enforcing Semantic Consistency for Cross Corpus Emotion Prediction using Adversarial Discrepancy Learning
期刊文章

Enforcing Semantic Consistency for Cross Corpus Emotion Prediction using Adversarial Discrepancy Learning

Chun-Min Chang, Gao-Yi ChaoChi-Chun Lee
IEEE Transactions on Affective Computing
2021

摘要

Acoustic distortion Correlation cross corpus learning Databases domain adaptation Emotion recognition generative adversarial network Nonlinear distortion semantic consistency Semantics speech emotion recognition Training Software Human-Computer Interaction
Mismatch between databases entails a challenge in performing emotion recognition on a practical-condition unlabeled database with labeled source data. The alignment between the source and target is crucial for conventional neural network; therefore, many studies have mapped two domains in a common feature space. However, the effect of distortion in emotion semantics across different conditions has been neglected in such work, and a sample from the target may be considered a high emotional annotation in the target but as low in the source. In this work, we propose the maximum regression discrepancy (MRD) network, which enforces semantic consistency in a source and target by adjusting the acoustic feature encoder to minimize discrepancy in maximally distorted samples through adversarial training. We show our framework in several experiments using three databases (the USC IEMOCAP, MSP-Improv, and MSP-Podcast) for cross corpus emotion prediction. Compared to the Source-only neural network and DANN, MRD network demonstrates a significant improvement between 5\\\\\\\\\\\\\\\\% and 10\\\\\\\\\\\\\\\\% in the concordance correlation coefficient (CCC) in cross-corpus prediction and between 3\\\\\\\\\\\\\\\\% and 10\\\\\\\\\\\\\\\\% for evaluation on MSP-PODCAST. We also visualize the effect of MRD on feature representation to shows the efficacy of the MRD structure we designed.

相關連結

指標

1 檢視次數

詳細資料

Logo image