融合时空特征的隧道场景跨相机车辆实时跟踪方法

doi:10.3778/j.issn.1002-8331.2307-0017

摘要/Abstract

摘要： 跨相机车辆跟踪对实现智慧交通具有重要意义，在隧道场景中，由于环境照度较低、同型车辆特征相似等因素的影响，现有目标重识别方案难以满足实际应用中对车辆跟踪精度和实时性的要求。考虑隧道交通场景下车辆的车型和时空特征，提出了一种融合时空特征的跨相机多目标跟踪方法。在YOLOv7目标检测模型中加入归一化注意力模块（NAM），使模型更关注感兴趣区域，结合相机标定获得车辆在真实空间中位置坐标。在卡尔曼滤波的基础上结合车辆速度进行目标位置预测，引入二次关联策略（BYTE）完成单相机下车辆跟踪，并使用间隔帧方法提高跟踪速度。提出以车型和时空特征为基础的跨相机目标匹配代价矩阵，采用匈牙利算法完成车辆目标匹配，从而实现跨相机车辆目标跟踪，生成隧道场景的车辆目标时空图。在构建的隧道场景跨相机车辆目标跟踪数据集上实验结果表明：跟踪准确度达到82.1%，检测跟踪整体速度达到115 FPS，跨相机目标匹配正确率达到94.9%，跟踪速度与精度优于其他方法。

关键词: 跨相机目标跟踪, 隧道场景, 时空特征, 注意力机制, 二次关联策略

Abstract: Cross-camera vehicle tracking is of great significance for realizing intelligent transportation. In the tunnel scene, the existing target re-identification scheme is difficult to meet the requirements of vehicle tracking accuracy and real-time in practical applications due to the influence of factors such as low environmental illumination and similar characteristics of the same type of vehicles. A cross-camera multi-target tracking method is proposed considering the vehicle type and spatiotemporal characteristics of vehicles in tunnel traffic scenarios. Firstly, the normalized attention module （NAM） is added to the YOLOv7 target detection model to make the model more focused on the region of interest, and combining with the camera calibration method, the vehicle position coordinates are obtained in real space. Secondly, the target position prediction is combined with vehicle velocity based on Kalman filtering, secondary correlation strategy（BYTE） is applied to complete single camera vehicle tracking, and interval frame method is used to improve the tracking speed. Finally, the cross-camera target matching cost matrix based on vehicle type and spatiotemporal characteristics is proposed, and the Hungarian algorithm is used to complete vehicle target matching, so as to realize cross-camera vehicle target tracking and generate the vehicle target spatiotemporal map of the tunnel scene. The experimental results on the cross-camera vehicle target tracking dataset constructed show that the tracking accuracy reaches 82.1%, the overall speed of detection and tracking reaches 115 frames per second, the accuracy of cross-camera target matching reaches 94.9%, and the tracking speed and accuracy are better than other methods.

Key words: cross-camera target tracking, tunnel scene, spatiotemporal feature, attention mechanism, secondary correlation strategy

苟铃滔, 宋焕生, 张朝阳, 文雅, 刘莅辰, 孙士杰. 融合时空特征的隧道场景跨相机车辆实时跟踪方法[J]. 计算机工程与应用, 2023, 59(24): 88-97.

GOU Lingtao, SONG Huansheng, ZHANG Zhaoyang, WEN Ya, LIU Lichen, SUN Shijie. Real-Time Cross-Camera Vehicle Tracking Method for Tunnel Scenes by Fusing Spatiotemporal Features[J]. Computer Engineering and Applications, 2023, 59(24): 88-97.

参考文献

[1] 刘彦.多相机协同的目标跟踪问题研究[D].太原：中北大学，2021.
LIU Y.Research on multi-camera cooperative target tracking[D].Taiyuan：North University of China，2021.
[2] HSU H M，HUANG T W，WANG G，et al.Multi-camera tracking of vehicles based on deep features RE-ID and trajectory-based camera link models[C]//Conference on Computer Vision and Pattern Recognition，2019：416-424.
[3] 刘艺，李蒙蒙，郑奇斌，等.视频目标跟踪算法综述[J].计算机科学与探索，2022，16（7）：1504-1515.
LIU Y，LI M M，ZHENG Q B，et al.Survey on video object tracking algorithms[J].Journal of Frontiers of Computer Science and Technology，2022，16（7）：1504-1515.
[4] 董美宝，杨涵文，郭文，等.多特征重检测的相关滤波无人机视觉跟踪[J].图学学报，2019，40（6）：1079-1086.
DONG M B，YANG H W，GUO W，et al.UVA visual tracking via multi-cue fusion correlation filter with redetection[J].Journal of Graphics，2019，40（6）：1079-1086.
[5] SHAO J，DU B，WU C，et al.Can we track targets from space? A hybrid kernel correlation filter tracker for satellite video[J].IEEE Transactions on Geoscience and Remote Sensing，2019，57（11）：8719-8731.
[6] HENRIQUES J F，CASEIRO R，MARTINS P，et al.High-speed tracking with kernelized correlation filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，37（3）：583-596.
[7] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//14th European Conference on Computer Vision，Amsterdam，The Netherlands，October 11-14，2016：21-37.
[8] DUAN K，BAI S，XIE L，et al.Centernet：keypoint triplets for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：6569-6578.
[9] CHU P，WANG J，YOU Q，et al.Transmot：spatial-temporal graph transformer for multiple object tracking[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision，2023：4870-4880.
[10] RISTANI E，TOMASI C.Features for multi-target multi-camera tracking and re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：6036-6046.
[11] ZHONG Z，ZHENG L，CAO D，et al.Re-ranking person re-identification with k-reciprocal encoding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1318-1327.
[12] WOJKE N，BEWLEY A，PAULUS D.Simple online and realtime tracking with a deep association metric[C]//2017 IEEE International Conference on Image Processing（ICIP），2017：3645-3649.
[13] BEWLEY A，GE Z，OTT L，et al.Simple online and realtime tracking[C]//2016 IEEE International Conference on Image Processing（ICIP），2016：3464-3468.
[14] DU Y，ZHAO Z，SONG Y，et al.Strongsort：make deepsort great again[J].arXiv：2202.13514，2022.
[15] ZHANG Y，WANG C，WANG X，et al.Fairmot：on the fairness of detection and re-identification in multiple object tracking[J].International Journal of Computer Vision，2021，129：3069-3087.
[16] ZHANG Y，SUN P，JIANG Y，et al.Bytetrack：multi-object tracking by associating every detection box[C]//17th European Conference on Computer Vision，Tel Aviv，Israel，October 23-27，2022.Cham：Springer Nature Switzerland，2022：1-21.
[17] HE Y，WEI X，HONG X，et al.Multi-target multi-camera tracking by tracklet-to-target assignment[J].IEEE Transactions on Image Processing，2020，29：5191-5205.
[18] HOU Y，DU H，ZHENG L.A locality aware city-scale multi-camera vehicle tracking system[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops，2019：167-174.
[19] XE D，JIANG Q，GU Y，et al.A kind of cross-camera tracking strategy in tunnel environment[C]//2022 41st Chinese Control Conference（CCC），2022：5390-5395.
[20] HE Z，LEI Y，BAI S，et al.Multi-camera vehicle tracking with powerful visual features and spatial-temporal cue[C]//Conference on Computer Vision and Pattern Recognition，2019：203-212.
[21] HE Y，HAN J，YU W，et al.City-scale multi-camera vehicle tracking by semantic attribute parsing and cross-camera tracklet matching[C]//Proceedings of the IEEE/CVF Workshops，2020：576-577.
[22] KANHERE N K，BIRCHFIELD S T.A taxonomy and analysis of camera calibration methods for traffic monitoring applications[J].IEEE Transactions on Intelligent Transportation Systems，2010，11（2）：441-452.
[23] WANG C Y，BOCHKOVSKIY A，LIAO H Y M.YOLOv7：trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[J].arXiv：2207.02696，2022.
[24] LIU Y，SHAO Z，TENG Y，et al.NAM：normalization-based attention module[J].arXiv：2111.12419，2021.
[25] WOO S，PARK J，LEE J Y，et al.Cbam：convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision，2018：3-19.
[26] TANG Z，NAPHADE M，LIU M Y，et al.Cityflow：a city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：8797-8806.
[27] AHARON N，ORFAIG R，BOBROVSKY B Z.BoT-SORT：robust associations multi-pedestrian tracking[J].arXiv：2206.
14651，2022.