Abstract:

Academic emotions can affect and regulate learners’ attention, memory, thinking and other cognitive activities. Automatic emotion recognition is the basis of emotion interaction and instructional decision in intelligent learning environment. At present, the research of emotion recognition mainly focuses on the recognition of discrete emotions, which is discontinuous in the timeline, and cannot accurately depict the evolution process of students’ academic emotions. In order to solve this problem, this paper establishes the dimensional emotional database of middle school students based on the crowd-sourcing method in the real online learning situation. And a deep learning analysis model based on continuous dimensional affective prediction is designed. In the experiment, learning materials that stimulate students’ academic emotions are identified according to students’ learning styles firstly. And then 32 experimenter are recruited for independent online learning and collecting real-time facial images. Next, dimensional database with 157 students’ academic emotion videos and 2 178 students’ facial expressions is obtained by the two-denationalization for each video emotion. Finally, a ConvLSTM net-based dimensional emotion model is established and tested on the dimensional emotion database for middle school students. The mean value of the Concordance Correlation Coefficient（CCC） is 0.581. Meanwhile, the mean value of the uniform correlation coefficient is 0.222 after the experiment on Aff-Wild database. The experiment shows that the dimension-based emotion model proposed in this paper improves the CCC correlation coefficient index by 7.6% to 43.0% in the dimension-based emotion recognition of Aff-Wild database.

Key words: continuous dimension emotion recognition, ConvLSTM, deep learning, academic emotion, dimensional emotion database

摘要：

学业情绪能够影响和调节学习者的注意、记忆、思维等认知活动，情绪自动识别是智慧学习环境中情感交互和教学决策的基础。目前情绪识别研究主要集中在离散情绪的识别，其在时间轴上是非连续的，无法精准刻画学生学业情绪演变过程，为解决这个问题，基于众包方法建立真实在线学习情境中的中学生学习维度情感数据集，设计基于连续维度情感预测的深度学习分析模型。实验中根据学生学习风格确定触发学生学业情绪的学习材料，并招募32位实验人员进行自主在线学习，实时采集被试面部图像，获取157个学生学业情绪视频；对每个视频进行情感Arousal和Valence二维化，建立包含2 178张学生面部表情的维度数据库；建立基于ConvLSTM网络的维度情感模型，并在面向中学生的维度情感数据库上进行实验，得到一致性相关系数（Concordance Correlation Coefficient，CCC）均值为0.581，同时在Aff-Wild公开数据集上进行实验，得到的一致相关系数均值为0.222。实验表明，提出的基于维度情感模型在Aff-Wild公开数据集维度情绪识别中CCC相关度系数指标提升了7.6%~43.0%。

关键词: 连续维度情感识别, ConvLSTM, 深度学习, 学业情绪, 维度情感数据库