权重推断与标签平滑的轻量级人脸表情识别

doi:10.3778/j.issn.1002-8331.2208-0344

摘要/Abstract

摘要： 针对轻量级网络在复杂环境下对面部表情的特征提取不够充分、模型参数存在冗余以及单标签数据集无法有效描述复杂情感倾向所带来的歧义表情等问题，提出了一种结合改进ShuffleNet与标签平滑学习的人脸表情识别方法。通过对原始网络的分析与剪裁，得到了改进后更紧凑的K5_Light_ShuffleNet，不仅优化了网络参数，还提高了模型的表征能力。为了增强模型对人脸表情图像局部细节特征的提取能力，抑制非表情特征，在模型中嵌入了设计的轻量化通道空间关键权重推断模块。通过标签平滑学习方法，在不引入额外信息的前提下，利用软标签分布监督网络的学习，以减少由于歧义表情对识别性能所带来的不利影响。实验结果表明，在公开的RAF-DB、AffectNet-7和AffectNet-8数据集上分别达到了86.91%、61.80%和58.75%的表情识别准确率，相较于目前其他人脸表情识别方法，其识别率有一定提高，同时模型参数量和计算量均保持在较低水平，利于其在实际中的应用。

关键词: 人脸表情识别, 关键权重推断, 轻量化, 标签平滑学习, 歧义表情

Abstract: Aiming at the problems of insufficient feature extraction of facial expressions by lightweight networks in complex environments, redundant model parameters, and the inability of single-label datasets to effectively describe the ambiguous expressions caused by complex emotional tendencies, a facial expression recognition method combining improved ShuffleNet and label smoothing learning is proposed. Firstly, by analyzing and tailoring the original network, the improved and more compact K5_Light_ShuffleNet is obtained, which not only optimizes the network parameters, but also improves the representation ability of the model. Secondly, in order to enhance the ability of the model to extract local detail features of facial expression images and suppress non-expression features, a designed lightweight channel-space key weight inference module is embedded in the model. Finally, through the label smoothing learning method, the learning of the network is supervised by the soft label distribution without introducing additional information, so as to reduce the adverse effect on the recognition performance due to ambiguous expressions. The experimental results show that the recognition accuracy rates of 86.91%, 61.80% and 58.75% are achieved on the RAF-DB, AffectNet-7 and AffectNet-8 datasets, respectively. Compared with other current facial expression recognition methods, the recognition rate has been improved to a certain extent. At the same time, the amount of model parameters and FLOPs are kept at a low level, which is beneficial to its practical application.

Key words: facial expression recognition, key weight inference, lightweight, label smoothing learning, ambiguous expression

刘劲, 罗晓曙, 徐照兴. 权重推断与标签平滑的轻量级人脸表情识别[J]. 计算机工程与应用, 2024, 60(2): 254-263.

LIU Jin, LUO Xiaoshu, XU Zhaoxing. Weight Inference and Label Smoothing for Lightweight Facial Expression Recognition[J]. Computer Engineering and Applications, 2024, 60(2): 254-263.

参考文献

[1] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[2] ZHOU N, LIANG R, SHI W. A lightweight convolutional neural network for real-time facial expression detection[J]. IEEE Access, 2020, 9: 5573-5584.
[3] KONG Y, REN Z, ZHANG K, et al. Lightweight facial expression recognition method based on attention mechanism and key region fusion[J]. Journal of Electronic Imaging, 2021, 30(6): 063002.
[4] NI J, ZHANG X, ZHANG J. Multiscale feature fusion attention lightweight facial expression recognition[J]. International Journal of Aerospace Engineering, 2022: 6523234.
[5] EKMAN P, FRIESEN W V. Facial action coding system: a technique for the measurement of facial actions[J]. Rivista Di Psichiatria, 1978, 47(2): 126-138.
[6] PLUTCHIK R. A general psychoevolutionary theory of emotion[J]. Social Science Information, 1982, 21(4/5): 529-553.
[7] ZHANG C B, JIANG P T, HOU Q, et al. Delving deep into label smoothing[J]. IEEE Transactions on Image Processing, 2021, 30: 5984-5996.
[8] ZHAO Z, LIU Q, ZHOU F. Robust lightweight facial expression recognition network with label distribution training[C]// Proceedings of the AAAI Conference on Artificial Intelligence, 2021: 3510-3519.
[9] MA N, ZHANG X, ZHENG H T, et al. Shufflenet v2: practical guidelines for efficient CNN architecture design[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 116-131.
[10] PENG C, ZHANG X, YU G, et al. Large kernel matters-improve semantic segmentation by global convolutional network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4353-4361.
[11] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19.
[12] LI S, DENG W. Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition[J]. IEEE Transactions on Image Processing, 2019, 28(1): 356-370.
[13] ALI M, BEHZAD H, MOHAMMAD H, et al. AffectNet: a new database for facial expression, valence, and arousal computation in the wild[J]. IEEE Transactions on Affective Computing, 2017, 10(1): 18-31.
[14] DENG J, GUO J, VERVERAS E, et al. Retinaface: single-shot multi-level face localisation in the wild[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 5203-5212.
[15] LI Y, ZENG J, SHAN S, et al. Occlusion aware facial expression recognition using CNN with attention mechanism[J]. IEEE Transactions on Image Processing, 2018, 28(5): 2439-2450.
[16] LI Y, LU Y, LI J, et al. Separate loss for basic and compound facial expression recognition in the wild[C]//Asian Conference on Machine Learning, 2019: 897-911.
[17] WANG K, PENG X, YANG J, et al. Region attention networks for pose and occlusion robust facial expression recognition[J]. IEEE Transactions on Image Processing, 2020, 99: 4057-4069.
[18] WANG K, PENG X, YANG J, et al. Suppressing uncertainties for large-scale facial expression recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 6897-6906.
[19] FARZANEH A H, QI X. Discriminant distribution-agnostic loss for facial expression recognition in the wild[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020: 406-407.
[20] 唐宏, 向俊玲, 陈海涛, 等. 多区域融合轻量级人脸表情识别网络[J]. 激光与光电子学进展, 2023, 60(6): 81-89.
TANG H, XIANG J L, CHEN H T, et al. Lightweight network based on multiregion fusion for facial expression recognition[J]. Laser & Optoelectronics Progress, 2023, 60(6): 81-89.
[21] MA H, CELIK T, LI H. Lightweight attention convolutional neural network through network slimming for robust facial expression recognition[J]. Signal, Image and Video Processing, 2021, 15(7): 1507-1515.
[22] HEWITT C, GUNES H. CNN-based facial affect analysis on mobile devices[J]. arXiv:1807.08775, 2018.
[23] SIQUEIRA H, MAGG S, WERMTER S. Efficient facial feature learning with wide ensemble-based convolutional neural networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 5800-5809.