Facial Expression Recognition Based on Multi-scale Feature Attention Mechanism

doi:10.3778/j.issn.1002-8331.2106-0174

Abstract

Abstract: Aiming at the problems that the effective feature extraction is not strong and the recognition accuracy is not high in the process of facial expression recognition with traditional convolutional neural network, a facial expression recognition method based on multi-scale feature attention mechanism is proposed. Firstly, a two-layer convolutional layer is used to extract shallow feature information. Secondly, the dilated convolution is added to the Inception structure parallelly to extract multi-scale feature information, and then the channel attention mechanism is introduced to improve model’s ability of expressing important feature information. Finally, it inputs the obtained features into the Softmax layer for classification. The simulation experimenal results show that the proposed model achieves 68.8% and 96.04% recognition accuracy on the public datasets FER2013 and CK+, respectively, which have better recognition performance than many classic algorithms.

Key words: convolutional neural network, facial expression recognition, dilated convolution, channel attention mechanism

摘要： 针对传统卷积神经网络在人脸表情识别过程中存在有效特征提取针对性不强、识别准确率不高的问题，提出一种基于多尺度特征注意力机制的人脸表情识别方法。用两层卷积层提取浅层特征信息；在Inception结构基础上并行加入空洞卷积，用来提取人脸表情的多尺度特征信息；引入通道注意力机制，提升模型对重要特征信息的表示能力；最后，将得到的特征输入Softmax层进行分类。通过在公开数据集FER2013和CK+上进行仿真实验，分别取得了68.8%和96.04%的识别准确率，结果表明该方法相比许多经典算法有更好的识别效果。

关键词: 卷积神经网络, 人脸表情识别, 空洞卷积, 通道注意力机制

ZHANG Peng, KONG Weiwei, TENG Jinbao. Facial Expression Recognition Based on Multi-scale Feature Attention Mechanism[J]. Computer Engineering and Applications, 2022, 58(1): 182-189.

张鹏, 孔韦韦, 滕金保. 基于多尺度特征注意力机制的人脸表情识别[J]. 计算机工程与应用, 2022, 58(1): 182-189.

References

[1] PANTIC M，ROTHKRANTZ L J M.Expert system for automatic analysis of facial expressions[J].Image and Vision Computing，2000，18（11）：881-905.
[2] LI T，DU C，NAREN T，et al.Using feature points and angles between them to recognize facial expressions by a neural network approach[J].IET Image Processing，2018，12（11）：1951-1955.
[3] BARTLETT M S，LITTLEWORK G，FASEL I，et al.Real time face detection and facial expression recognition：development and applications to human computer interaction[C]//Proceedings of 2003 Conference on Computer Vision and Pattern Recognition Workshop，2003：53.
[4] WOLD S，ESBENSEN K，GELADI P.Principal component analysis[J].Chemometrics and Intelligent Laboratory Systems，1987，2（1/3）：37-52.
[5] OJALA T，PIETIKAINEN M，MAENPAA T.Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2002，24（7）：971-987.
[6] DALAL N，TRIGGS B.Histograms of oriented gradients for human detection[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition（CVPR），2005：886-893.
[7] ZHU Y，LI X，WU G.Face expression recognition based on equable principal component analysis and linear regression classification[C]//Proceedings of 2016 3rd International Conference on Systems and Informatics（ICSAI），2016：876-880.
[8] HE Y，HE X.Facial expression recognition based on multi-feature fusion and HOSVD[C]//Proceedings of 2019 IEEE 3rd Information Technology，Networking，Electronic and Automation Control Conference（ITNEC），2019：638-643.
[9] HINTON G E，SALAKHUTDINOV R R.Reducing the dimensionality of data with neural networks[J].Science，2006，313：504-507.
[10] KRIZHEVSKY A，SUTSKEVER I，HINTON G E，et al.ImageNet classification with deep convolutional neutral networks[J].Communications of the ACM，2017，60（6）：84-90.
[11] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：1-9.
[12] SALUNKE V V，PATIL C G.A new approach for automatic face emotion recognition and classification based on deep networks[C]//Proceedings of 2017 International Conference on Computing，Communication，Control and Automation，2017：1-5.
[13] ZHOU Y，FENG Y Y，ZENG S Y，et al.Facial expression recognition based on convolutional neutral network[C]//Proceedings of 2019 IEEE 10th International Conference on Software Engineering and Service Science（ICSESS），2019：410-413.
[14] LI B，LIMA D.Facial expression recognition via ResNet-50[J].International Journal of Cognitive Computing in Engineering，2021，2：57-64.
[15] AGRAWAL A，MITTAL N.Using CNN for facial expression recognition：A study of the effects of kernel size and number of filters on accuracy[J].The Visual Computer，2020，36（2）：405-412.
[16] HU J，LI S，SUN G，et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（8）：2011-2023.
[17] LI J，JIN K，ZHOU D L，et al.Attention mechanism-based CNN for facial expression recognition[J].Neurocomputing，2020，411：340-350.
[18] 赖振意，陈人和，钱育蓉.结合空洞卷积的CNN实时微表情识别算法[J].计算机应用研究，2020，37（12）：3777-3780.
LAI Z Y，CHEN R H，QIAN Y R.Real-time micro-expression recognition algorithm based on atrous convolutions for CNN[J].Application Research of Computers，2020，37（12）：3777-3780.
[19] 张焯林，赵建伟，曹飞龙.构建带空洞卷积的深度神经网络重建高分辨率图像[J].模式识别与人工智能，2019，32（3）：259-267.
ZHANG Z L，ZHAO J W，CAO F L.Building deep neural networks with dilated convolutions to reconstruct high-resolution image[J].Pattern Recognition and Artificial Intelligence，2019，32（3）：259-267.
[20] WU H，ZOU Z，GUI J，et al.Multi-grained attention networks for single image super-resolution[J].IEEE Transactions on Circuits and Systems for Video Technology，2020，31（2）：512-522.
[21] LUCEY P，COHN J F，KANADE T，et al.The extended cohn-kanade dataset（CK+）：A complete dataset for action unit and emotion-specified expression[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Regression-Workshops，2010：94-101.
[22] 徐琳琳，张树美，赵俊莉.构建并行卷积神经网络的表情识别算法[J].中国图象图形学报，2019，24（2）：227-236.
XU L L，ZHANG S M，ZHAO J L.Expression recognition algorithm for parallel convolutional neural networks[J].Journal of Image and Graphics，2019，24（2）：227-236.
[23] MOLLAHOSSEINI A，DAVID C，MAHOOR M H.Going deeper in facial expression recognition using deep neural networks[C]//Proceedings of 2016 IEEE Winter Conference on Applications of Computer Vision（WACV），2016：1-10.
[24] 钱勇生，邵洁，季欣欣，等.基于改进卷积神经网络的多视角人脸表情识别[J].计算机工程与应用，2018，54（24）：12-19.
QIAN Y S，SHAO J，JI X X，et al.Multi-view facial expression recognition based on improved convolutional neural network[J].Computer Engineering and Applications，2018，54（24）：12-19.
[25] 李勇，林小竹，蒋梦莹.基于跨连接LeNet-5网络的面部表情识别[J].自动化学报，2018，44（1）：176-182.
LI Y，LIN X Z，JIANG M Y.Facial expression recognition with cross-connect LeNet-5 network[J].Acta Automatica Sinica，2018，44（1）：176-182.
[26] LI J N，LAM E Y.Facial expression recognition using deep neural networks[C]//Proceedings of 2015 IEEE International Conference on Imaging Systems and Techniques（IST），2015：1-6.
[27] ZENG N，ZHANG H，SONG B，et al.Facial expression recognition via learning deep sparse autoencoders[J].Neurocomputing，2018，273：643-649.
[28] SUN X，LV M.Facial expression recognition based on a hybrid model combining deep and shallow features[J].Cognitive Computation，2019，11（4）：587-597.