关键词: 情感计算, 情感标签标注, 脑电图, 多模态融合

Abstract: Emotion tagging is one field of interest in affective computing. Several works are published on this topic focusing on emotion tagging for images, audios and multimedia clips. In this paper the importance of audio signals are analyzed under previous proposed EEG-based brain encoding emotion tagging framework. The open accessed affective computing dataset DEAP is employed as the benchmark. For the analysis, three kinds of visual features and one set of audio features are extracted from video clips. The visual features are used for emotion tagging under the proposed framework at first then the combination of audio and visual features are used through the same procedure. The results indicate that emotion tagging accuracies are improved by combining audio and visual features compared with accuracy using only visual features. Moreover, no performance loss is caused by the increasement of feature dimensions.

Key words: affective computing, emotion tagging, electroencephalogram（EEG）, multimodal fusion