Wearing Mask Pedestrian Tracking Based on Improved YOLOv7 and DeepSORT

doi:10.3778/j.issn.1002-8331.2210-0479

Abstract

Abstract: A pedestrian tracking algorithm based on improved YOLOv7 and DeepSORT is proposed to solve the problem that whether pedestrians wear masks cannot be correctly judged due to face occlusion and missed detection in video sequences. The algorithm combines mask detection, pedestrian detection and tracking. Firstly, by adding attention mechanism to the backbone network of YOLOv7, shallow feature maps are added to enhance the network’s ability to perceive small targets and improve the accuracy of mask detection and pedestrian detection. Secondly, the intra-frame relationship module uses the Hungarian algorithm to correlate the intra-frame targets and mark the mask wearing of pedestrians. Then, the direction difference factor is added to the association cost of the DeepSORT algorithm to eliminate the inconsistency between the historical detection direction and the new detection speed direction of the tracking trajectory. Finally, the improved DeepSORT algorithm is used to track pedestrians and update the mask wearing mark for each track, achieve tracking of pedestrians wearing masks and those not wearing masks. The experimental results show that the average detection accuracy mAP50 of the improved YOLOv7 network is 3.83 percentage points higher than that of the original algorithm. On the MOT16 dataset, the tracking accuracy MOTA of this algorithm is 17.1 percentage points higher than that of DeepSORT algorithm, and the tracking precision MOTP is increased by 2.6% percentage points. Compared with the detection algorithm, this algorithm can track more pedestrians whether wearing masks, and has better results.

Key words: pedestrians tracking, mask detection, attention mechanism

摘要： 针对视频序列中因脸部遮挡、漏检而造成的无法正确判断行人是否佩戴口罩的问题，提出一种基于改进YOLOv7与DeepSORT的佩戴口罩行人跟踪算法。该算法将口罩检测、行人检测与跟踪相结合，通过在YOLOv7的主干网络中添加注意力机制，增加浅层特征图，加强网络对小目标的感知能力，提高口罩检测与行人检测精度；帧内关系模块利用匈牙利算法进行帧内目标关联，对行人进行口罩佩戴标记；将方向差因素加入到DeepSORT算法的关联代价中，消除跟踪轨迹的历史预测方向和新检测速度方向不一致问题；使用改进的DeepSORT算法对行人进行跟踪，并对每条轨迹进行口罩佩戴标记更新，实现对佩戴口罩与未佩戴口罩行人的跟踪。实验结果表明，改进的YOLOv7网络平均检测精度mAP50相比原始算法提升了3.83个百分点；在MOT16数据集上，该算法的跟踪准确性MOTA相较DeepSORT算法提高了17.1个百分点，跟踪精度MOTP提高了2.6个百分点。与检测算法相比，提出的算法能够跟踪到更多的行人是否佩戴了口罩，具有更好的效果。

关键词: 行人跟踪, 口罩检测, 注意力机制

ZHAO Yuanlong, SHAN Yugang, YUAN Jie. Wearing Mask Pedestrian Tracking Based on Improved YOLOv7 and DeepSORT[J]. Computer Engineering and Applications, 2023, 59(6): 221-230.

赵元龙, 单玉刚, 袁杰. 改进YOLOv7与DeepSORT的佩戴口罩行人跟踪[J]. 计算机工程与应用, 2023, 59(6): 221-230.

References

[1] 宋瑶，刘俊.改进U-Net的新冠肺炎图像分割方法[J].计算机工程与应用，2021，57（19）：243-251.
SONG Y，LIU J.Improved U-Net network for COVID-19 image segmentation[J].Computer Engineering and Applications，2021，57（19）：243-251.
[2] 王欣然，田启川，张东.人脸口罩佩戴检测研究综述[J].计算机工程与应用，2022，58（10）：13-26.
WANG X R，TIAN Q C，ZHANG D.Review of research on face mask wearing detection[J].Computer Engineering and Applications，2022，58（10）：13-26.
[3] BOCHKOVSKIY A，WANG C Y，LIAO H.YOLOv4：optimal speed and accuracy of object detection[J].arXiv：2004.
10934，2020.
[4] WANG C Y，BOCHKOVSKIY A，LIAO H.YOLOv7：trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL].（2022-07-02）[2022-10-26].https：//arxiv.org/pdf/2207.02696v1.pdf.
[5] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//Proceedings of the European Conference on Computer Vision，Netherlands，Oct 10-16，2016.Berlin，Heidelberg：Springer-Verlag，2016：21-37.
[6] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conferenceon Computer Vision，Chile，Dec 13-16，2015.New York：IEEE，2015：1440-1448.
[7] REN S Q，HE K M，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（6）：1137-1149.
[8] FANG K，XIANG Y，LI X，et al.Recurrent autoregressive networks for online multi-object tracking[C]//Proceedings of the Winter Conference on Applications of Computer Vision，Lake Tahoe，USA，Mar 12-15，2018.New York：IEEE，2018：466-475.
[9] WOJKE N，BEWLEY A，PAULUS D.Simple online and realtime tracking with a deep association metric[C]//Proceedings of the IEEE International Conference on Image Processing，Beijing，Sep 17-20，2017.Piscataway：IEEE，2017：3645-3649.
[10] ZHANG Y，WANG C，WANG X，et al.FairMOT：on the fairness of detection and re-identification in multiple object tracking[EB/OL].（2020-04-08）[2022-10-26].https：//arxiv.org/pdf/2004.01888.pdf.
[11] 张修宝，林子原，田万鑫，等.全天候自然场景下的人脸佩戴口罩识别技术[J].中国科学：信息科学，2020，50（7）：1110-1120.
ZHANG X B，LIN Z Y，TIAN W X，et al.Mask-wearing recognition in the wild[J].Scientia Sinica：Informationis，2020，50（7）：1110-1120.
[12] 牛作东，覃涛，李捍东，等.改进RetinaFace的自然场景口罩佩戴检测算法[J].计算机工程与应用，2020，56（12）：1-7.
NIU Z D，QIN T，LI H D，et al.Improved algorithm of RetinaFace for natural scene mask wear detection[J].Computer Engineering and Applications，2020，56（12）：1-7.
[13] 陈昭俊，储珺，曾伦杰.基于动态加权类别平衡损失的多类别口罩佩戴检测[J].图学学报，2022，43（4）：590-598.
CHEN Z J，CHU J，ZENG L J.Multi category mask wearing detection based on dynamic weighted category balance loss[J].Journal of Graphics，2022，43（4）：590-598.
[14] 毛晓波，徐向阳，李楠，等.基于改进SSD和Jetson Nano的口罩佩戴检测门禁系统[J].郑州大学学报（工学版），2021，42（6）：85-92.
MAO X B，XU X Y，LI N，et al.A mask wear detection access control system based on improved SSD and Jetson Nano[J].Journal of Zhengzhou University（Engineering Science），2021，42（6）：85-92.
[15] BEWLEY A，GE Z，OTT L，et al.Simple online and realtime tracking[C]//Proceedings of the IEEE International Conference on Image Processing，Phoenix，Sep 25-28，2016.New York，IEEE，2016：3464-3468.
[16] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//Proceedings of the IEEE European Conference on Computer Vision，Munich，Germany，Sep 8-14，2018.Berlin，Heidelberg：Springer-Verlag，2018：3-19.
[17] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，Seattle，Jun 13-19，2020.New York，IEEE，2020：11531-11539.
[18] 姚彤，王军，孟丽华，等.基于非接触式人体测量的东北地区女青年人体比例研究[J].大连工业大学学报，2014，33（3）：226-228.
YAO T，WANG J，MENG L H，et al.Research on northeast young females’ body proportion based on non-contact 3D body measurement[J].Journal of Dalian Polytechnic University，2014，33（3）：226-228.
[19] CAO J，WENG X，KHIRODKAR R，et al.Observation-centric SORT：rethinking SORT for robust multi-object tracking[EB/OL].（2020-04-08）[2022-10-26].https：//arxiv.org/pdf/2203.14360.pdf.
[20] MILAN A，LEAL-TAIXé L，REID I，et al.MOT16：a benchmark for multi-object tracking[J].arXiv：1603.00831，2016.
[21] YU F，LI W，LI Q，et al.POI：multiple object tracking with high performance detection and appearance feature[C]//Proceedings of the European Conference on Computer Vision，Netherlands，Oct 10-16，2016.Berlin，Heidelberg：Springer-Verlag，2016：36-42.