Lightweight Masked Face Recognition Algorithm Incorporating Attention Mechanism

doi:10.3778/j.issn.1002-8331.2206-0222

Abstract

Abstract: The globalization pandemic of COVID-19 has made wearing masks to become a norm in people’s lives, and this preventive measure brings new challenges to face recognition algorithms. To address this problem, this paper proposes a lightweight masked face recognition algorithm. Firstly, this network introduces GhostNet as the backbone feature extraction network to improve the recognition speed. Secondly a FocusNet feature enhancement extraction network incorporating spatial attention mechanism is proposed to make the model focus on the upper half of the face region that are not covered by masks. Then, in order to overcome the problem of inadequate masked face dataset, a data augmentation method using 3D face mesh is proposed to add face mask. Finally, the experimental results show that, compared with the benchmark model, the proposed model reduces the number of model parameters by 84%, while the AP of masked face recognition increases by 4.29?percentage points, which better balances speed and accuracy.

Key words: masked face recognition, attention model, 3D face mesh generation

摘要： COVID-19的全球化大流行使得佩戴口罩出行成为人们生活中的常态，这种防疫措施给人脸识别算法带来了新的挑战。针对这一问题，提出了一种口罩遮挡下的轻量级人脸识别算法，该算法改进GhostNet为主干特征提取网络；提出了融合空间注意力机制的FocusNet特征加强提取网络，使模型聚焦于未被口罩遮挡的上半脸区域；针对当前口罩遮挡人脸数据集不充分的问题，提出了一种采用三维人脸网络生成添加口罩遮挡的数据增强方法。实验表明，所提出的改进模型与基准模型相比，模型参数量降低84%的同时，戴口罩人脸的识别率提升4.29个百分点，较好地平衡了速度与精度。

关键词: 戴口罩人脸识别, 注意力机制, 三维人脸网格生成

YE Zixun, ZHANG Hongying, HE Yujun. Lightweight Masked Face Recognition Algorithm Incorporating Attention Mechanism[J]. Computer Engineering and Applications, 2023, 59(3): 166-174.

叶子勋, 张红英, 何昱均. 融合注意力机制的轻量级戴口罩人脸识别算法[J]. 计算机工程与应用, 2023, 59(3): 166-174.

References

[1] TAM S Y，TAM V C W，LAW H K W，et al.Rationale for mass masking in controlling the COVID-19 pandemic[J].Frontiers in Public Health，2021：2296-2565.
[2] WANG M，DENG W.Deep face recognition：a survey[J].Neurocomputing，2021，429：215-244.
[3] SCHROFF F，KALENICHENKO D，PHILBIN J.FaceNet：a unified embedding for face recognition and clustering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：815-823.
[4] DENG J，GUO J，XUE N，et al.ArcFace：additive angular margin loss for deep face recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2019：4690-5274.
[5] WANG H.CosFace：large margin cosine loss for deep face recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2018：5265-5274.
[6] HUANG G B，MATTAR M，BERG T，et al.Labeled faces in the wild：a database for studying face recognition in unconstrained environments[C]//Workshop on Faces in ‘Real-Life’ Images：Detection，Alignment，and Recognition，2008.
[7] JIA H，MARTINEZ A M.Support vector machines in face recognition with occlusions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2009：136-141.
[8] WU C Y，DING J J.Occluded face recognition using low-rank regression with generalized gradient direction[J].Pattern Recognition，2018，80：256-268.
[9] MONTERO D，NIETO M，LESKOVSKY P，et al.Boosting masked face recognition with multi-task arcface[J].arXiv：2104.09874，2021.
[10] FENG T，XU L，YUAN H，et al.Towards mask-robust face recognition[C]//IEEE Conference on Computer Vision（ICCV），2021：1492-1496.
[11] LI Y，GUO K，LU Y，et al.Cropping and attention based approach for masked face recognition[J].Applied Intelligence，2021：3012-3025.
[12] NETO P C，BOUTROS F，PINTO J R，et al.My eyes are up here：promoting focus on uncovered regions in masked face recognition[J].arXiv：2108.00996，2021.
[13] DING F，PENG P，HUANG Y，et al.Masked face recognition with latent part detection[C]//Proceedings of the 28th ACM International Conference on Multimedia，2020：2281-2289.
[14] CAI J，HAN H，CUI J，et al.Semi-supervised natural face deocclusion[J].IEEE Transactions on Information Forensics and Security，2020，16：1044-1057.
[15] GENG M，PENG P，HUANG Y，et al.Masked face recognition with generative data augmentation and domain constrained ranking[C]//Proceedings of the 28th ACM International Conference on Multimedia，2020：2246-2254.
[16] DIN N U，JAVED K，BAE S，et al.A novel GAN-based network for unmasking of masked face[J].IEEE Access，2020，8：44276-44287.
[17] MARTíNEZ-DíAZ Y，MéNDEZ-VáZQUEZ H，LUEVANO L S，et al.Towards accurate and lightweight masked face recognition：an experimental evaluation[J].IEEE Access，2021，10：7341-7353.
[18] DENG J K，GUO J，AN X，et al.Masked face recognition challenge：the insightFace track report[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2021：1437-1444.
[19] WANG J，LIU Y，HU Y，et al.Facex-zoo：a pytorch toolbox for face recognition[J].arXiv：2101.04407，2021.
[20] HAN K，WANG Y，TIAN Q，et al.Ghostnet：more features from cheap operations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2020：1580-1589.
[21] HOWARD A G，ZHU M，CHEN B，et al.MobileNets：efficient convolutional neural networks for mobile vision applications[J].arXiv：1704.04861，2017.
[22] SANDLER M，HOWARD A，ZHU M，et al.Mobilenetv2：inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2018：4510-4520.
[23] HOWARD A，SANDLER M，CHU G，et al.Searching for mobilenetv3[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），2019：1314-1324.
[24] WANG C Y，LIAO H Y M，WU Y H，et al.CSPNet：a new backbone that can enhance learning capability of CNN[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2020：390-391.
[25] FENG Y，WU F，SHAO X，et al.Joint 3d face reconstruction and dense alignment with position map regression network[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：534-551.
[26] KARTYNNIK Y，ABLAVATSKI A，GRISHCHENKO I，et al.Real-time facial surface geometry from monocular video on mobile GPUs[J].arXiv：1907.06724，2019.
[27] YI D，LEI Z，LIAO S，et al.Learning face representation from scratch[J].arXiv：1411.7923，2014.
[28] WANG Z，WANG G，HUANG B，et al.Masked face recognition dataset and application[J].arXiv：2003.09093，2020.
[29] FU R，HU Q，DONG X，et al.Axiom-based Grad-CAM：towards accurate visualization and explanation of CNNs[J].arXiv：2008.02312，2020.
[30] SPRINGENBERG J T，DOSOVITSKIY A，BROX T，et al.Striving for simplicity：the all convolutional net[J].arXiv：1412.6806，2014.