Facial Expression Recognition Method Using Optimized Pruning GoogLeNet

doi:10.3778/j.issn.1002-8331.2102-0296

Abstract

Abstract:

In order to improve the accuracy of facial expression recognition and speed up the processing, a facial expression recognition method based on optimized pruning GoogLeNet is proposed. Firstly, the GoogLeNet network is used to extract facial features, in which the concept module can deepen the learning depth, and the typical classifier is used to realize facial expression classification. Then, the global maximum pooling layer is added to improve the GoogLeNet network, and the location information of the detected target is retained. The sigmoid cross entropy is used as the training target to obtain the comprehensive facial expression feature information. Finally, the pruning algorithm is used to train the GoogLeNet network, pruning the low weight connection and retraining the network, so as to simplify the network structure and parameters and improve the operation efficiency. The proposed method is tested on the data sets of JAFFE, CK+ and Cohn Kanade. The results show that the recognition accuracy of the proposed method is 83.84%, 85.09% and 84.87% respectively, and the running time is less than 200?ms, which is better than the comparison method and has certain universality.

Key words: pruning algorithm, GoogLeNet, facial expression recognition, Inception module, global maximum pooling layer, operating efficiency

摘要：

为了提高人脸表情识别的准确率和加快处理速度，提出了一种基于优化剪枝GoogLeNet的人脸表情识别方法。利用GoogLeNet网络提取面部特征，其中Inception模块加深学习深度，并利用典型的分类器实现人脸表情分类。改进GoogLeNet网络，添加全局最大池化层并保留检测目标的位置信息，以Sigmoid交叉熵作为训练目标，获得全面的人脸表情特征信息。通过剪枝算法对GoogLeNet网络进行训练、修剪低权重连接和再训练网络等操作，以简化网络结构和参数量，提高运行效率。在JAFFE、CK+和Cohn-Kanade数据集上对所提方法进行验证，实验结果表明，所提方法的识别准确率分别为83.84%、85.09%和84.87%，运行时间低于200?ms，优于对比方法，具有较好的适用性。

关键词: 剪枝算法, GoogLeNet, 人脸表情识别, Inception模块, 全局最大池化层, 运行效率

ZHANG Hongli, BAI Xiangyu. Facial Expression Recognition Method Using Optimized Pruning GoogLeNet[J]. Computer Engineering and Applications, 2021, 57(19): 179-188.

张宏丽，白翔宇. 利用优化剪枝GoogLeNet的人脸表情识别方法[J]. 计算机工程与应用, 2021, 57(19): 179-188.

References

[1] 张人，何宁.微表情识别研究综述[J].计算机工程与应用，2021，57（1）：38-47.
ZHANG R，HE N.A survey of micro-expression recognition methods[J].Computer Engineering and Applications，2021，57（1）：38-47.
[2] YAO Y，HUANG D，YANG X，et al.Texture and geometry scattering representation-based facial expression recognition in 2D+3D videos[J].ACM Transactions on Multimedia Computing Communications and Applications，2018，14：1-23.
[3] 张爱梅，徐杨.注意力分层双线性池化残差网络的表情识别[J].计算机工程与应用，2020，56（23）：161-166.
ZHANG A M，XU Y.Attention hierarchical bilinear pooling residual network for expression recognition[J].Computer Engineering and Applications，2020，56（23）：161-166.
[4] 林克正，白婧轩，李昊天，等.深度学习下融合不同模型的小样本表情识别[J].计算机科学与探索，2020，14（3）：127-137.
LIN K Z，BAI J X，LI H T，et al.Facial expression recognition with small samples fused with different models under deep learning[J].Journal of Frontiers of Computer Science & Technology，2020，14（3）：127-137.
[5] CHEN J，XU R，LIU L.Deep peak-neutral difference feature for facial expression recognition[J].Multimedia Tools and Applications，2018，77：29871-29887.
[6] 谭论正，丁锐，夏利民.结合归一化正交投影与CNN的表情识别[J].小型微型计算机系统，2019，40（10）：2221-2226.
TAN L Z，DING Y，XIA L M.Facial expression recognition combined with orthogonal neighborhood preserving projectio-nand CNN[J].Journal of Chinese Computer Systems，2019，40（10）：2221-2226.
[7] 李婷婷，胡玉龙，魏枫林.基于GAN改进的人脸表情识别算法及应用[J].吉林大学学报（理学版），2020，58（3）：163-168.
LI T T，HU Y L，WEI F L.Improved facial expression recognition algorithm based on GAN and application[J].Journal of Jilin University（Science Edition），2020，58（3）：163-168.
[8] 刘全明，辛阳阳.端到端的低质人脸图像表情识别[J].小型微型计算机系统，2020，41（3）：668-672.
LIU Q M，XIN Y Y.Face expression recognition based on end-to-end low-quality face images[J].Journal of Chinese Computer Systems，2020，41（3）：668-672.
[9] JAN A，DING H，MENG H，et al.Accurate facial parts localization and deep learning for 3D facial expression recognition[C]//Proceedings of the 13th IEEE International Conference on Automatic Face and Gesture Recognition，2018：466-472.
[10] ISLAM B，HOSSAIN A.Fusion of features and extreme learning machine for facial expression recognition[J].Journal of Computer ENCES，2019，15（12）：1833-1841.
[11] 黎明，彭秀姣，王艳.基于改进的字典学习与稀疏表示的人脸表情识别[J].系统仿真学报，2018，30（1）：28-35.
LI M，PENG X J，WANG Y.Facial expression recognition based on improved dictionary learning and sparse representation[J].Journal of System Simulation，2018，30（1）：28-35.
[12] TAMFOUS A B，DRIRA H，AMOR B B.Sparse coding of shape trajectories for facial expression and action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（10）：2594-2607.
[13] 刘芾，李茂军，胡建文，等.基于低像素人脸图像的表情识别[J].激光与光电子学进展，2020，57（10）：97-104.
LIU F，LI M J，HU J W，et al.Expression recognition based on low pixel face images[J].Laser & Optoelectronics Progress，2020，57（10）：97-104.
[14] 王晓红，梁祐慈，麻祥才.一种基于Inception思想的人脸表情分类深度学习算法研究[J].光学技术，2020，46（3）：94-100.
WANG X H，LIANG Y C，Ma X C.Facial expression classification algorithm research based on ideology of inception[J].Optical Technique，2020，46（3）：94-100.
[15] DU L，HU H.Weighted patch-based manifold regularization dictionary pair learning model for facial expression recognition using iterative optimization classification strategy[J].Computer Vision and Image Understanding，2019，18（4）：13-24.
[16] KEYU Y，WENGMING Z，ZHEN C，et al.Unsupervised facial expression recognition using domain adaptation based dictionary learning approach[J].Neuro Computing，2018，319（11）：84-91.
[17] LIANG D，LIANG H，YU Z，et al.Deep convolutional BiLSTM fusion network for facial expression recognition[J].The Visual Computer，2020，36（3）：499-508.
[18] 陈亮，吴攀，刘韵婷.面向类内差距表情的深度学习识别[J].中国图象图形学报，2020，25（4）：679-687.
CHEN L，WU P，LIU Y T.Depth learning recognition method for intra-class gap expression[J].Journal of Image and Graphics，2020，25（4）：679-687.
[19] MUHAMMAD N A，NASIR A A，IBRAHIM Z，et al.Evaluation of CNN，alexnet and GoogleNet for fruit recognition[J].Indonesian Journal of Electrical Engineering and Computer Science，2018，12（2）：468-475.
[20] JIN X，WU L，LI X，et al.ILGNet：Inception modules with connected local and global features for efficient image aesthetic quality classification using domain adaptation[J].IET Computer Vision，2019，13（2）：206-212.
[21] 胡敏，余胜男，王晓华.基于约束性循环一致生成对抗网络的人脸表情识别方法[J].电子测量与仪器学报，2019，33（4）：174-182.
HU M，YU S N，WANG X H.Facial expression recognition method based on constraint cycle-consistent generative adversarial networks[J].Journal of Electronic Measurement and Instrumentation，2019，33（4）：174-182.
[22] 张继威，牛少彰，曹志义，等.基于深度学习和约束稀疏表达的人脸识别算法[J].北京理工大学学报，2019，39（3）：255-261.
ZHANG J W，NIU S Z，CAO Z Y，et al.Face recognition via deep learning and constraint sparse representation[J].Transactions of Beijing Institute of Technology，2019，39（3）：255-261.
[23] JAYASIMHA Y，REDDY R V S.A facial expression recognition model using hybrid feature selection and support vector machines[J].International Journal of Information and Computer Security，2020，13（3）：1-14.
[24] 郭蓓，达飞鹏.基于局部特征的表情不变3维人脸识别算法[J].计算机辅助设计与图形学学报，2019，31（7）：1086-1094.
GUO B，DA F P.Expression-Invariant 3D face recognition based on local descriptors[J].Journal of Computer-Aided Design & Computer Graphics，2019，31（7）：1086-1094.
[25] LILIANA D Y，BASARUDDIN T，WIDYANTO M R，et al.Fuzzy Emotion：A natural approach to automatic facial expression recognition from psychological perspective using fuzzy system[J].Cognitive Processing，2019，20（4）：391-403.