Review of Research on Face Mask Wearing Detection

doi:10.3778/j.issn.1002-8331.2110-0396

Abstract

Abstract: Face mask wearing detection is an emerging research topic that has developed rapidly in the past two years in the context of the global COVID-19 epidemic. Under regular epidemic situation, wearing masks is an important means of effective epidemic prevention, therefore it is essential to remind and check people whether to wear masks in public places. Using artificial intelligence to complete mask wearing detection can achieve the purpose of real-time supervision, save human resources and effectively avoid mistakes, missed detection and other problems. The models and relevant algorithms used in current mask wearing detection research are reviewed. Firstly, the task and application background of mask wearing detection are described. Then, the detection algorithms based on deep neural networks and object detection models are summarized and analyzed, the advantages and disadvantages, improvement methods and application scenarios of different research schemes are discussed. Secondly, common related data sets are introduced, and the detection performance of each algorithm is compared. Finally, the existing problems and the direction of future development are discussed and prospected.

Key words: mask wearing detection, deep neural networks, object detection

摘要： 人脸口罩佩戴检测是近两年在全球新冠疫情背景下快速发展的一个新兴研究课题。疫情常态下，佩戴口罩是有效防疫的重要手段，因此公共场所下对人员是否佩戴口罩的检查与提醒必不可少。利用人工智能完成口罩佩戴检测工作可以达到实时监督的目的，节省人力资源，有效避免误检、漏检等问题。对当前口罩佩戴检测研究所使用的网络模型和相关算法进行了详细梳理。针对口罩佩戴检测任务及其应用背景进行了简要说明；重点总结和分析了基于深度神经网络和基于目标检测模型两种思路的检测算法，主要讨论了不同研究方案的优缺点、改进方法和适用场景；介绍了常用的相关数据集，对比展现了各算法检测性能；对仍然存在的问题以及未来发展的方向进行了探讨和展望。

关键词: 口罩佩戴检测, 深度神经网络, 目标检测

WANG Xinran, TIAN Qichuan, ZHANG Dong. Review of Research on Face Mask Wearing Detection[J]. Computer Engineering and Applications, 2022, 58(10): 13-26.

王欣然, 田启川, 张东. 人脸口罩佩戴检测研究综述[J]. 计算机工程与应用, 2022, 58(10): 13-26.

References

[1] 中华预防医学会新型冠状病毒肺炎防控专家组.新型冠状病毒肺炎流行病学特征的最新认识[J].中国病毒病杂志，2020，10（2）：86-92.
Special Expert Group for Control of the Epidemic of Novel Coronavirus Pneumonia of the Chinese Preventive Medicine Association.An update on the epidemiological characteristics of novel coronavirus pneumonia（COVID-19）[J].Chinese Journal of Viral Diseases，2020，10（2）：86-92.
[2] GARG P S.Face mask detection system using deep learning[J].International Journal for Modern Trends in Science and Technology，2020，6（12）：161-164.
[3] 王远大.UCloud开放人脸口罩检测服务借助AI算法加快疫情防控[J].通信世界，2020（5）：33-34.
WANG Y D.UCloud open face mask detection service uses AI algorithm to speed up epidemic prevention and control[J].Communications World，2020（5）：33-34.
[4] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.ImageNet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems 25：26th Annual Conference on Neural Information Processing Systems，2012：1097-1105.
[5] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv：1409.
1556，2014.
[6] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[7] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition，2015：1-9.
[8] IOFFE S，SZEGEDY C.Batch normalization：accelerating deep network training by reducing internal covariate shift[C]//32nd International Conference on Machine Learning，2015：448-456.
[9] SZEGEDY C，VANHOUCKE V，IOFFE S，et al.Rethinking the Inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：2818-2826.
[10] MULLER R，KORNBLITH S，HINTON G.When does label smoothing help?[J].arXiv：1906.02629，2019.
[11] SZEGEDY C，IOFFE S，VANHOUCKE V，et al.Inception-v4，Inception-ResNet and the impact of residual connections on learning[C]//21st AAAI Conference on Artificial Intelligence，2017.
[12] OUMINA A，EL MAKHFI N，HAMDI M.Control the COVID-19 pandemic：face mask detection using transfer learning[C]//2020 IEEE 2nd International Conference on Electronics，Control，Optimization and Computer Science，2020：1-5.
[13] CHOLLET F.Xception：deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017.
[14] SANDLER M，HOWARD A，ZHU M，et al.MobileNetv2：inverted residuals and linear bottlenecks[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：4510-4520.
[15] PLATT J C.Sequential minimal optimization：a fast algorithm for training support vector machines：MSR-TR-98-14[R].Microsoft Research，1998.
[16] ABEYWICKRAMA T，CHEEMA M A，TANIAR D.K-nearest neighbors on road networks：a journey in experimentation and in-memory implementation[J].arXiv：1601.01549，2016.
[17] LOEY M，MANOGARAN G，TAHA M H N，et al.A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic[J].Measurement，2021，167：108288.
[18] DIETTERICH T G.Ensemble learning[J].The Handbook of Brain Theory and Neural Networks，2002，2（1）：110-125.
[19] 刘国明，江巨浪，查兵，等.基于深度神经网络的口罩佩戴检测[J].安庆师范大学学报（自然科学版），2021，27（2）：54-58.
LIU G M，JIANG J L，ZHA B，et al.Mask wearing detection based on deep neural network[J].Journal of Anqing Normal University（Natural Science Edition），2021，27（2）：54-58.
[20] GATHANI J，SHAH K.Detecting masked faces using region-based convolutional neural network[C]//2020 IEEE 15th International Conference on Industrial and Information Systems，2020：156-161.
[21] 刘启刚，孙向阳，徐伟.针对实时场景的口罩检测模型设计[J].实验技术与管理，2021，38（8）：76-81.
LIU Q G，SUN X Y，XU W.Mask detection model design for real-time scene[J].Experimental Technology and Management，2021，38（8）：76-81.
[22] CHOWDARY G J，PUNN N S，SONBHADRA S K，et al.Face mask detection using transfer learning of Inceptionv3[C]//8th International Conference on Big Data Analytics.Cham：Springer，2020：81-90.
[23] LIN M，CHEN Q，YAN S.Network in network[J].arXiv：1312.4400，2013.
[24] HINTON G E，SRIVASTAVA N，KRIZHEVSKY A，et al.Improving neural networks by preventing co-adaptation of feature detectors[J].arXiv：1207.0580，2012.
[25] 金映谷，张涛，杨亚宁，等.基于MobileNet V2的口罩佩戴识别研究[J].大连民族大学学报，2021，23（5）：404-409.
JIN Y G，ZHANG T，YANG Y N，et al.Mask wearing recognition based on MobileNet V2[J].Journal of Dalian Minzu University，2021，23（5）：404-409.
[26] ALAWI A E B，QASEM A M.Lightweight CNN-based models for masked face recognition[C]//2021 International Congress of Advanced Technology and Engineering，2021：1-5.
[27] 许德刚，王露，李凡.深度学习的典型目标检测算法研究综述[J].计算机工程与应用，2021，57（8）：10-25.
XU D G，WANG L，LI F.Review of typical object detection algorithms for deep learning[J].Computer Engineering and Applications，2021，57（8）：10-25.
[28] 肖雨晴，杨慧敏.目标检测算法在交通场景中应用综述[J].计算机工程与应用，2021，57（6）：30-41.
XIAO Y Q，YANG H M.Research on application of object detection algorithm in traffic scene[J].Computer Engineering and Applications，2021，57（6）：30-41.
[29] 张开华，樊佳庆，刘青山.视觉目标跟踪十年研究进展[J].计算机科学，2021，48（3）：40-49.
ZHANG K H，FAN J Q，LIU Q S.Advances on visual object tracking in past decade[J].Computer Science，2021，48（3）：40-49.
[30] 胡正平，张乐，李淑芳，等.视频监控系统异常目标检测与定位综述[J].燕山大学学报，2019，43（1）：1-12.
HU Z P，ZHANG L，LI S F，et al.Research on abnormal target detection and location in video surveillance system[J].Journal of Yanshan University，2019，43（1）：1-12.
[31] 赵文清，孔子旭，周震东，等.增强小目标特征的航空遥感目标检测[J].中国图象图形学报，2021，26（3）：644-653.
ZHAO W Q，KONG Z X，ZHOU Z D，et al.Target detection algorithm of aerial remote sensing based on feature enhancement technology[J].Journal of Image and Graphics，2021，26（3）：644-653.
[32] 罗会兰，陈鸿坤.基于深度学习的目标检测研究综述[J].电子学报，2020，48（6）：1230-1239.
LUO H L，CHEN H K.Survey of object detection based on deep learning[J].Acta Electronica Sinica，2020，48（6）：1230-1239.
[33] GIRSHICK R，DONAHUE J，DARRELL T，et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition，2014：580-587.
[34] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//14th European Conference on Computer Vision.Cham：Springer，2016：21-37.
[35] LIN T Y，GOYAL P，GIRSHICK R，et al.Focal loss for dense object detection[C]//2017 IEEE International Conference on Computer Vision，2017：2980-2988.
[36] GIRSHICK R.Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision，2015：1440-1448.
[37] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems 28：Annual Conference on Neural Information Processing Systems，2015：91-99.
[38] SHYLAJA H N，LATHA H N，POORNIMA H N，et al.Detection and localization of mask occluded faces by transfer learning using Faster RCNN[C]//2021 International Conference on Innovative Computing & Communication，2021.
[39] 任钰，刘全金，黄忠，等.基于Faster R-CNN与迁移学习的口罩佩戴检测算法[J].安庆师范大学学报（自然科学版），2021，27（4）：25-30.
REN Y，LIU Q J，HUANG Z，et al.Mask wearing detection algorithm based on Faster R-CNN and transfer learning[J].Journal of Anqing Normal University（Natural Science Edition），2021，27（4）：25-30.
[40] 李泽琛，李恒超，胡文帅，等.多尺度注意力学习的Faster R-CNN口罩人脸检测模型[J].西南交通大学学报，2021，56（5）：1002-1010.
LI Z C，LI H C，HU W S，et al.Masked face detection model based on multi-scale attention-driven Faster R-CNN[J].Journal of Southwest Jiaotong University，2021，56（5）：1002-1010.
[41] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//15th European Conference on Computer Vision.Cham：Springer，2018：3-19.
[42] 万子伦，张彦波，王多峰，等.复杂环境下多任务识别的人脸口罩检测算法[J].微电子学与计算机，2021，38（10）：21-27.
WAN Z L，ZHANG Y B，WANG D F，et al.Face mask detection algorithm for multitask recognition in complex environment[J].Microelectronics & Computer，2021，38（10）：21-27.
[43] REDMON J，FARHADI A.YOLOv3：an incremental improvement[J].arXiv：1804.02767，2018.
[44] BOCHKOVSKIY A，WANG C Y，LIAO H Y M.YOLOv4：optimal speed and accuracy of object detection[J].arXiv：2004.10934，2020.
[45] 王艺皓，丁洪伟，李波，等.复杂场景下基于改进YOLOv3的口罩佩戴检测算法[J].计算机工程，2020，46（11）：12-22.
WANG Y H，DING H W，LI B，et al.Mask wearing detection algorithm based on improved YOLOv3 in complex scenes[J].Computer Engineering，2020，46（11）：12-22.
[46] WANG C Y，LIAO H Y M，WU Y H，et al.CSPNet：a new backbone that can enhance learning capability of CNN[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops，2020：390-391.
[47] HE K，ZHANG X，REN S，et al.Spatial pyramid pooling in deep convolutional networks for visual recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，37（9）：1904-1916.
[48] 曾成，蒋瑜，张尹人.基于改进YOLOv3的口罩佩戴检测方法[J].计算机工程与设计，2021，42（5）：1455-1462.
ZENG C，JIANG Y，ZHANG Y R.Improved YOLOv3 detection algorithm for mask wearing[J].Computer Engineering and Design，2021，42（5）：1455-1462.
[49] 张路达，邓超.多尺度融合的YOLOv3人群口罩佩戴检测方法[J].计算机工程与应用，2021，57（16）：283-290.
ZHANG L D，DENG C.Multi-scale fusion of YOLOv3 crowed mask wearing detection method[J].Computer Engineering and Applications，2021，57（16）：283-290.
[50] 孙世丹，郑佳春，赵世佳，等.基于YOLO改进算法的安全帽和口罩佩戴自动同时检测[J].集美大学学报（自然科学版），2021，26（4）：379-384.
SUN S D，ZHENG J C，ZHAO S J，et al.Safety helmet and mask wearing automatic simultaneous detection based on YOLO improved algorithm[J].Journal of Jimei University（Natural Science），2021，26（4）：379-384.
[51] 曹城硕，袁杰.基于YOLO-Mask算法的口罩佩戴检测方法[J].激光与光电子学进展，2021，58（8）：211-218.
CAO C S，YUAN J.Mask wearing detection method based on YOLO-Mask[J].Laser & Optoelectronics Progress，2021，58（8）：211-218.
[52] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[53] 程可欣，王玉德.基于改进YOLOv3的自然场景人员口罩佩戴检测算法[J].计算机系统应用，2021，30（2）：231-236.
CHENG K X，WANG Y D.Masks worn by people in natural scenes based on improved YOLOv3 detection algorithm[J].Computer?Systems?&?Applications，2021，30（2）：231-236.
[54] REZATOFIGHI H，TSOI N，GWAK J Y，et al.Generalized intersection over union：a metric and a loss for bounding box regression[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：658-666.
[55] 管军霖，智鑫.基于YOLOv4卷积神经网络的口罩佩戴检测方法[J].现代信息科技，2020，4（11）：9-12.
GUAN J L，ZHI X.Mask wearing detection method based on YOLOv4 convolutional neural network[J].Modern Information Technology，2020，4（11）：9-12.
[56] 冉鹏飞.复杂光照下基于深度学习的佩戴口罩检测[J].自动化与仪表，2021，36（4）：67-72.
RAN P F.Mask wearing detection based on deep learning in complex illumination[J].Automation & Instrumentation，2021，36（4）：67-72.
[57] 谈世磊，别雄波，卢功林，等.基于YOLOv5网络模型的人员口罩佩戴实时检测[J].激光杂志，2021，42（2）：147-150.
TAN S L，BIE X B，LU G L，et al.Real-time detection for mask-wearing of personnel based on YOLOv5 network model[J].Laser Journal，2021，42（2）：147-150.
[58] 肖博健，万烂军，陈俊权.采用YOLOv5模型的口罩佩戴识别研究[J].福建电脑，2021，37（3）：35-37.
XIAO B J，WAN L J，CHEN J Q.Research on mask wearing recognition using YOLOv5 model[J].Journal of Fujian Computer，2021，37（3）：35-37.
[59] 叶子勋，张红英.YOLOv4口罩检测算法的轻量化改进[J].计算机工程与应用，2021，57（17）：157-168.
YE Z X，ZHANG H Y.Lightweight improvement of YOLOv4 mask detection algorithm[J].Computer Engineering and Applications，2021，57（17）：157-168.
[60] HOWARD A，SANDLER M，CHU G，et al.Searching for MobileNetv3[C]//2019 IEEE/CVF International Conference on Computer Vision，2019：1314-1324.
[61] 罗禹杰，张剑，陈亮，等.基于自适应空间特征融合的轻量化目标检测算法设计[J].激光与光电子学进展，2022，59（4）：310-320.
LUO Y J，ZHANG J，CHEN L，et al.Design of lightweight target detection algorithm based on adaptive spatial feature fusion[J].Laser & Optoelectronics Progress，2022，59（4）：310-320.
[62] LIU S，HUANG D，WANG Y.Learning spatial fusion for single-shot object detection[J].arXiv：1911.09516，2019.
[63] 丁培，阿里甫·库尔班，耿丽婷，等.自然环境下实时人脸口罩检测与规范佩戴识别[J].计算机工程与应用，2021，57（24）：268-275.
DING P，ALIFU K，GENG L T，et al.Real-time face mask detection and standard wearing recognition method in natural environment[J].Computer Engineering and Applications，2021，57（24）：268-275.
[64] 王兵，乐红霞，李文璟，等.改进YOLO轻量化网络的口罩检测算法[J].计算机工程与应用，2021，57（8）：62-69.
WANG B，LE H X，LI W J，et al.Mask detection algorithm based on improved YOLO lightweight network[J].Computer Engineering and Applications，2021，57（8）：62-69.
[65] 朱杰，王建立，王斌.基于YOLOv4-tiny改进的轻量级口罩检测算法[J].液晶与显示，2021，36（11）：1525-1534.
ZHU J，WANG J L，WANG B.Improved lightweight mask detection algorithm based on YOLOv4-tiny[J].Chinese Journal of Liquid Crystals and Displays，2021，36（11）：1525-1534.
[66] 叶茂，马杰，王倩，等.多尺度特征融合的轻量化口罩佩戴检测算法[J/OL].计算机工程（2021-10-15）[2021-11-13].https：//doi.org/10.19678/j.issn.1000-3428.0062231.
YE M，MA J，WANG Q，et al.Lightweight mask wearing detection algorithm with multi-scale feature fusion[J/OL].Computer Engineering（2021-10-15）[2021-11-13].https：//doi.org/10.19678/j.issn.1000-3428.0062231.
[67] 彭成，张乔虹，唐朝晖，等.基于YOLOv5增强模型的口罩佩戴检测方法研究[J/OL].计算机工程（2021-07-29）[2021-09-12].https：//doi.org/10.19678/j.issn.1000-3428.0061502.
PENG C，ZHANG Q H，TANG C H，et al.A face mask wearing detection method based on YOLOv5 enhancement model[J/OL].Computer Engineering（2021-07-29）[2021-09-12].https：//doi.org/10.19678/j.issn.1000-3428.0061502.
[68] LIN T Y，DOLLAR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：2117-2125.
[69] 李雨阳，沈记全，翟海霞，等.基于改进SSD的口罩佩戴检测算法[J/OL].计算机工程（2021-09-09）[2021-11-20].https：//doi.org/10.19678/j.issn.1000-3428.0062150.
LI Y Y，SHEN J Q，ZHAI H X，et al.Mask wearing detection algorithm based on improved SSD[J/OL].Computer Engineering（2021-09-09）[2021-11-20].https：//doi.org/10.
19678/j.issn.1000-3428.0062150.
[70] 阮士峰.基于改进SSD算法的行人佩戴口罩检测研究[J].科技经济导刊，2020，28（35）：9-13.
YUAN S F.Detection of pedestrian wearing masks based on improved SSD algorithm[J].Technology and Economic Guide，2020，28（35）：9-13.
[71] NAGRATH P，JAIN R，MADAN A，et al.SSDMNV2：a real time DNN-based face mask detection system using single shot multibox detector and MobileNetV2[J].Sustainable Cities and Society，2021，66：102692.
[72] 毛晓波，徐向阳，李楠，等.基于改进SSD和Jetson Nano的口罩佩戴检测门禁系统[J].郑州大学学报（工学版），2021，42（6）：85-92.
MAO X B，XU X Y，LI N，et al.Mask wearing detection algorithm based on improved SSD[J/OL].Journal of Zhengzhou University（Engineering Science），2021，42（6）：85-92.
[73] 邓黄潇.基于迁移学习与RetinaNet的口罩佩戴检测的方法[J].电子技术与软件工程，2020（5）：209-211.
DENG H X.Methods of mask wearing detection based on transfer learning with RetinaNet[J].Electronic Technology & Software Engineering，2020（5）：209-211.
[74] DENG J，GUO J，ZHOU Y，et al.RetinaFace：single-stage dense face localisation in the wild[J].arXiv：1905.00641，2019.
[75] 牛作东，覃涛，李捍东，等.改进RetinaFace的自然场景口罩佩戴检测算法[J].计算机工程与应用，2020，56（12）：1-7.
NIU Z D，QIN T，LI H D，et al.Improved algorithm of RetinaFace for natural scene mask wear detection[J].Computer Engineering and Applications，2020，56（12）：1-7.
[76] JIANG M，FAN X，YAN H.Retina facemask：a face mask detector[J].arXiv：2005.03950，2020.
[77] YANG S，LUO P，LOY C C，et al.Wider face：a face detection benchmark[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：5525-5533.
[78] GE S，LI J，YE Q，et al.Detecting masked faces in the wild with LLE-CNNs[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：2682-2690.
[79] WANG Z，WANG G，HUANG B，et al.Masked face recognition dataset and application[J].arXiv：2003.09093，2020.
[80] CABANI A，HAMMOUDI K，BENHABILES H，et al.MaskedFace-Net a dataset of correctly/incorrectly masked face images in the context of COVID-19[J].Smart Health，2021，19：100144.
[81] ZHU Z，HUANG G，DENG J，et al.WebFace260M：a benchmark unveiling the power of million-scale deep face recognition[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：10492-10502.
[82] HUANG G B，MATTAR M，BERG T，et al.Labeled faces in the wild：a database for studying face recognition in unconstrained environments[C]//Workshop on Faces in Real-Life Images：Detection，Alignment，and Recognition，2008.
[83] 刘淇缘，卢树华，兰凌强.遮挡人脸检测方法研究进展[J].计算机工程与应用，2020，56（13）：33-46.
LIU Q Y，LU S H，LAN L Q.Research progress on occluded face detection methods[J].Computer Engineering and Applications，2020，56（13）：33-46.
[84] 聂永琦，曹慧，杨锋，等.深度学习在糖尿病视网膜病灶检测中的应用综述[J].计算机工程与应用，2021，57（20）：25-41.
NIE Y Q，CAO H，YANG F，et al.Review of application of deep learning in detection of diabetic retinal lesions[J].Computer Engineering and Applications，2021，57（20）：25-41.
[85] 许虞俊，李晨.基于YOLO优化的轻量级目标检测网络[J].计算机科学，2021，48（S2）：265-269.
XU Y J，LI C.Light-weight object detection network optimized based on YOLO family[J].Computer Science，2021，48（S2）：265-269.
[86] 徐遐龄，刘涛，田国辉，等.有遮挡环境下的人脸识别方法综述[J].计算机工程与应用，2021，57（17）：46-60.
XU X L，LIU T，TIAN G H，et al.Review of occlusion face recognition methods[J].Computer Engineering and Applications，2021，57（17）：46-60.