Face Recognition Method Based on Improved Visual Transformer

doi:10.3778/j.issn.1002-8331.2208-0182

Abstract

Abstract: Most face recognition methods rely on convolutional neural networks currently, which construct cascaded multi-layer processing units and fuse local features with convolution operation, result in ignoring the global semantic information and attention to the key areas of the face image. This paper proposes a face recognition method based on improved visual Transformer. Shuffle Transformer is introduced as the backbone network of feature extraction, the global information of feature map is captured through self-attention mechanism and Shuffle operation, and the long-distance dependence relationship is established between feature points to enhance the feature perception ability of the model. At the same time, considering the characteristics of ArcFace loss function and center loss function, the fusion loss is designed as the objective function, which utilizes the intra-class constraints to enlarge the angle interval and increase the discrimination of feature space. The proposed method achieves average accuracy of 99.83%, 95.87%, 90.05%, 98.05% and 97.23% on five challenging benchmark face datasets, LFW, CALFW, CPLFW, AGEDB-30 and CFP. It is proved that the improved model can effectively promote the ability of face feature extraction, and achieve better recognition effect than that of convolutional neural network in the same scale.

Key words: face recognition, visual Transformer, self-attention mechanism, ArcFace loss function

摘要： 目前大多数人脸识别方法依赖于卷积神经网络，通过级联的形式构建多层处理单元，利用卷积操作融合局部特征，忽略了人脸全局语义信息，缺乏对人脸重点区域的关注度。针对上述问题，提出一种基于改进视觉Transformer的人脸识别方法，引入Shuffle Transformer作为特征提取骨干网络，通过自注意力机制以及Shuffle操作捕捉特征图全局信息，建立特征点之间的长距离依赖关系，提高模型的特征感知能力；同时，结合ArcFace损失函数和中心损失函数的特点，设计融合损失作为目标函数，利用类内约束扩大角度间隔，提高特征空间的辨别性。该方法在LFW、CALFW、CPLFW、AgeDB-30和CFP五个具有挑战性的基准测试人脸数据集上分别取得了99.83%、95.87%、90.05%、98.05%、97.23%的平均准确率，能够有效提升人脸特征提取能力，识别效果优于同等规模卷积神经网络。

关键词: 人脸识别, 视觉Transformer, 自注意力机制, ArcFace损失函数

JI Ruirui, XIE Yuhui, LUO Fengkai, MEI Yuan. Face Recognition Method Based on Improved Visual Transformer[J]. Computer Engineering and Applications, 2023, 59(8): 117-126.

季瑞瑞, 谢宇辉, 骆丰凯, 梅远. 改进视觉Transformer的人脸识别方法[J]. 计算机工程与应用, 2023, 59(8): 117-126.

References

[1] SUNDARARAJAN K，WOODARD D L.Deep learning for biometrics：a survey[J].ACM Computing Surveys，2018，51（3）：1-34.
[2] ZENG D，VELDHUIS R，SPREEUWERS L.A survey of face recognition techniques under occlusion[J].IET Biometrics，2021，10（6）：581-606.
[3] ADJABI I，OUAHABI A，BENZAOUI A，et al.Past，present，and future of face recognition：a review[J].Electronics，2020，9（8）：1188.
[4] TAIGMAN Y，YANG M，RANZATO M A，et al.Deepface：closing the gap to human-level performance in face verification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2014：1701-1708.
[5] SUN Y，WANG X，TANG X.Deep learning face representation from predicting 10，000 classes[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2014：1891-1898.
[6] SUN Y，CHEN Y，WANG X，et al.Deep learning face representation by joint identification-verification[C]//Advances in Neural Information Processing Systems（NIPS），2014.
[7] SUN Y，WANG X，TANG X.Deeply learned face representations are sparse，selective，and robust[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：2892-2900.
[8] SUN Y，LIANG D，WANG X，et al.Deepid3：face recognition with very deep neural networks[J].arXiv：1502.00873，2015.
[9] SCHROFF F，KALENICHENKO D，PHILBIN J.Facenet：a unified embedding for face recognition and clustering[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：815-823.
[10] DU H，SHI H，LIU Y，et al.Semi-siamese training for shallow face learning[C]//European Conference on Computer Vision（ECCV），2020.
[11] LI S，XU J，XU X，et al.Spherical confidence learning for face recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2021.
[12] HUANG Z，ZHANG J，SHAN H.When age-invariant face recognition meets face age synthesis：a multi-task learning framework[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2021.
[13] 罗岩，冯天波，邵洁.基于注意力及视觉Transformer的野外人脸表情识别[J].计算机工程与应用，2022，58（10）：200-207.
LUO Yan，FENG Tianbo，SHAO Jie.Facial expression recognition in wild based on attention and vision Transformer[J].Computer Engineering and Applications，2022，58（10）：200-207.
[14] LIU Z，LIN Y，CAO Y，et al.Swin transformer：hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2021：10012-10022.
[15] HUANG Z，BEN Y，LUO G，et al.Shuffle transformer：rethinking spatial shuffle for vision transformer[J].arXiv：2106.03650，2021.
[16] HOWARD A G，ZHU M，CHEN B，et al.MobileNets：efficient convolutional neural networks for mobile vision applications[J].arXiv：1704.04861，2017.
[17] DENG J，GUO J，XUE N，et al.Arcface：additive angular margin loss for deep face recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2019：4690-4699.
[18] WEN Y，ZHANG K，LI Z，et al.A discriminative feature learning approach for deep face recognition[C]//European Conference on Computer Vision（ECCV）.Cham：Springer，2016：499-515.
[19] GUO Y，ZHANG L，HU Y，et al.Ms-celeb-1m：a dataset and benchmark for large-scale face recognition[C]//European Conference on Computer Vision（ECCV）.Cham：Springer，2016：87-102.
[20] HUANG G B，MATTAR M，BERG T，et al.Labeled faces in the wild：a database for studying face recognition in unconstrained environments[C]//Workshop on Faces in “Real-Life” Images：Detection，Alignment，and Recognition，2008.
[21] ZHENG T，DENG W，HU J.Cross-age lfw：a database for studying cross-age face recognition in unconstrained environments[J].arXiv：1708.08197，2017.
[22] ZHENG T，DENG W.Cross-pose LFW：a database for studying cross-pose face recognition in unconstrained environments[J].arXiv：1708.08197，2017.
[23] MOSCHOGLOU S，PAPAIOANNOU A，SAGONAS C，et al.AgeDB：the first manually collected，in-the-wild age database[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops（CVPR），2017：51-59.
[24] SENGUPTA S，CHEN J C，CASTILLO C，et al.Frontal to profile face verification in the wild[C]//2016 IEEE Winter Conference on Applications of Computer Vision（WACV），2016：1-9.
[25] WANG J，LIU Y，HU Y，et al.Facex-zoo：a pytorch toolbox for face recognition[C]//Proceedings of the 29th ACM International Conference on Multimedia，2021：3779-3782.
[26] WANG F，XIANG X，CHENG J，et al.Normface：L2 hypersphere embedding for face verification[C]//Proceedings of the 25th ACM International Conference on Multimedia，2017：1041-1049.
[27] PARKHI O M，VEDALDI A，ZISSERMAN A.Deep face recognition[C]//British Machine Vision Conference（BMVC），2015.
[28] LIU J，DENG Y，BAI T，et al.Targeting ultimate accuracy：face recognition via deep embedding[J].arXiv：1506.07310，2015.
[29] ZHOU E，CAO Z，YIN Q.Naive-deep face recognition：touching the limit of LFW benchmark or not?[J].arXiv：1501.04690，2015.
[30] LIU W，WEN Y，YU Z，et al.Sphereface：deep hypersphere embedding for face recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：212-220.
[31] HASNAT A，BOHNé J，MILGRAM J，et al.Deepvisage：making face recognition simple yet with powerful generalization skills[C]//Proceedings of the IEEE International Conference on Computer Vision Workshops，2017：1682-1691.
[32] WANG F，CHENG J，LIU W，et al.Additive margin softmax for face verification[J].IEEE Signal Processing Letters，2018，25（7）：926-930.