Expression Recognition with Separable Convolution Channel Enhancement Features

doi:10.3778/j.issn.1002-8331.2008-0076

Abstract

Abstract: At present, facial expression recognition network model has low recognition rate and complex parameters. To mitigate this problem, a facial expression recognition method with separable convolution channel enhancement features is proposed in this paper. First, a lightweight convolutional neural network structure is designed to extract facial features, and depthwise separable convolution is used in the convolutional layer to reduce network parameters. Then, the squeeze-and-excitation module is introduced to assign weights to the features of different channels, and different squeeze rates are used in different convolutional layers to enhance the network’s ability to extract facial expressions. Finally, the extracted features are fed into a classifier to achieve facial expression classification, and experiments are carried out and analyzed on CK+ and FER2013 dataset. The experimental results show that, compared with the existing methods, the recognition rate of the network structure proposed in this paper increases by 0.15 percentage points and 3.29 percentage points respectively on the CK+ and FER2013 dataset, and the number of network model parameters decreases by 75%. The proposed method not only reduces the network parameters, but also improves the accuracy of facial expression recognition.

Key words: facial expression, convolutional neural network, depthwise separable convolution, squeeze-and-excitation module

摘要： 针对目前人脸表情识别准确率不高、网络模型参数复杂等问题，提出一种增强可分离卷积通道特征的人脸表情识别研究方法。设计了一种轻量型卷积神经网络结构提取表情特征，在卷积层中采用深度可分离卷积减少网络参数；引入了压缩激发模块，对不同通道的特征进行权重分配，在不同的卷积层采用不同的压缩率来增强网络对人脸表情的特征提取能力；将提取到的特征送入分类器实现人脸表情分类，在CK+和FER2013数据集上进行实验并分析。实验结果表明：与现有方法相比，提出的网络结构在CK+和FER2013数据集上，识别率分别提高了0.15个百分点和3.29个百分点，且网络模型参数量降低了75%。所提方法在降低网络参数的同时，提高了表情识别准确率。

关键词: 人脸表情, 卷积神经网络, 深度可分离卷积, 压缩激发模块

LIANG Huagang, LEI Yixiong. Expression Recognition with Separable Convolution Channel Enhancement Features[J]. Computer Engineering and Applications, 2022, 58(2): 184-192.

梁华刚, 雷毅雄. 增强可分离卷积通道特征的表情识别研究[J]. 计算机工程与应用, 2022, 58(2): 184-192.

References

[1] 阮凯，邱卫根.多信息融合的深度学习人脸表情识别算法研究[J].计算机工程与应用，2019，55（5）：192-196.
RUAN Kai，QIU Weigen.Study on facial expression recognition algorithm of multi-information fusion based on deep learning[J].Computer Engineering and Applications，2019，55（5）：192-196.
[2] SUN N，CHEN Z，DAY R.Facial expression recognition using digitalised facial features based on active shape model[C]//Sixth International Conference on Computer Science，Engineering & Applications，2016.
[3] 梁华刚，易生，茹锋.结合像素模式和特征点模式的实时表情识别[J].中国图象图形学报，2017，22（12）：1737-1749.
LIANG Huagang，YI Sheng，RU Feng.Real-time expression recognition method based on pixel and feature point patterns[J].Journal of Image and Graphics，2017，22（12）：1737-1749.
[4] 张哲源，张灵，陈云华.结合分块LBP与投影字典对学习的表情识别[J].计算机工程与应用，2019，55（12）：149-154.
ZHANG Zheyuan，ZHANG Ling，CHEN Yunhua.Facial expression recognition combined with block LBP and projective dictionary pair learning[J].Computer Engineering and Applications，2019，55（12）：149-154.
[5] 梁华刚，张志伟，王亚茹.自适应Gabor卷积核编码网络的表情识别方法[J].计算机工程与应用，2020，56（10）：149-156.
LIANG Huagang，ZHANG Zhiwei，WANG Yaru.Expression recognition method for adaptive gabor convolution kernelcoding network[J].Computer Engineering and Applications，2020，56（10）：149-156.
[6] LECUN Y，BOTTOU L.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE，1998，86（11）：2278-2324.
[7] KRIZHEVSKY A，SUTSKEVER I，HINTON G E，et al.ImageNet classification with deep convolutional neural networks[C]//Neural Information Processing Systems，2012：1097-1105.
[8] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition，2014.
[9] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//IEEE Conference on Computer Vision and Pattern Recognition，2015：1-9.
[10] 王建霞，陈慧萍，李佳泽，等.基于多特征融合卷积神经网络的人脸表情识别[J].河北科技大学学报，2019，40（6）：540-547.
WANG Jianxia，CHEN Huiping，LI Jiaze，et al.Facial expression recognition based on multi-feature fusion convolution network[J].Journal of Hebei University of Science and Technology，2019，40（6）：540-547.
[11] 杜进，陈云华，张灵，等.基于改进深度残差网络的低功耗表情识别[J].计算机科学，2018，45（9）：303-307.
DU Jin，CHEN Yunhua，ZHANG Ling，et al.Energy-efficient facial expression recognition based on improved deep residual networks[J].Computer Science，2018，45（9）：303-307.
[12] 吕诲，童倩倩，袁志勇.基于人脸分割的复杂环境下表情识别实时框架[J].计算机工程与应用，2020，56（12）：134-140.
LV Hui，TONG Qianqian，YUAN Zhiyong.Real time architecture for facial expression recognition in complex scenes based on face region segmentation[J].Computer Engineering and Applications，2020，56（12）：134-140.
[13] HU J，SHEN L，ALBANIE S，et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（8）：2011-2023.
[14] IOFFE S，SZEGEDY C.Batch normalization：accelerating deep network training by reducing internal covariate shift[C]//32nd International Conference on Machine Learning，2015.
[15] KLAMBAUER G，UNTERTHINER T，MAYR A，et al.Self-normalizing neural networks[C]//31st Conference on Neural Information Processing Systems，2017.
[16] CHOLLET F.Xception：deep learning with depthwise separable convolutions[C]//IEEE Conference on Computer Vision and Pattern Recognition，2017：1800-1807.
[17] GOODFELLOW I J，ERHAN D，CARRIER P L，et al.Challenges in representation learning：a report on three machine learning contests[J].Neural Netw，2015，64：59-63.
[18] LUCEY P，COHN J F，KANADE T，et al.The extended Cohn-Kanade dataset（CK+）：a complete dataset for action unit and emotion-specified expression[C]//IEEE Conference on Computer Vision and Pattern Recognition，2010.
[19] OUELLET S.Real-time emotion recognition for gaming using deep convolutional network features[J].arXiv：1408.
3750，2014.
[20] SZEGEDY C，IOFFE S，VANHOUCKE V，et al.Inception-v4，Inception-resNet and the impact of residual connections on learning[C]//National Conference on Artificial Intelligence，2016：4278-4284.
[21] 徐琳琳，张树美，赵俊莉.构建并行卷积神经网络的表情识别算法[J].中国图象图形学报，2019，24（2）：227-236.
XU Linlin，ZHANG Shumei，ZHAO Junli.Expression recognition algorithm for parallel convolutional neural networks[J].Journal of Image and Graphics，2019，24（2）：227-236.
[22] AGRAWAL A，MITTAL N.Using CNN for facial expression recognition：a study of the effects of kernel size and number of filters on accuracy[J].The Visual Computer，2020，36（2）：405-412.
[23] FERNANDEZ P D M，PEA F A G，REN T I，et al.FERAtt：facial expression recognition with attention net[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops（CVPRW），2020.
[24] DING H，ZHOU S K，CHELLAPPA R，et al.FaceNet2ExpNet：regularizing a deep face recognition net for expression recognition[C]//IEEE International Conference on Automatic Face Gesture Recognition，2017：118-126.
[25] 孙晓，丁小龙.基于生成对抗网络的人脸表情数据增强方法[J].计算机工程与应用，2020，56（4）：115-121.
SUN Xiao，DING Xiaolong.Data augmentation method based on generative adversarial networks for facial expressionrecognition sets[J].Computer Engineering and Applications，2020，56（4）：115-121.