广视角特征融合记忆网络的多目标跟踪算法

doi:10.3778/j.issn.1002-8331.2410-0484

摘要/Abstract

摘要： 基于无人机视频的多目标跟踪是一项重要的视觉任务，具有广泛的应用前景。然而，由于无人机视角范围广、远距离小尺寸目标难以追踪且目标运动迅速，传统方法面临诸多挑战。为此，提出了一种基于Transformer技术的多目标跟踪方法，称为WideTrack。该方法设计了广视角特征融合记忆网络，以增强对远距离小尺寸目标的捕捉能力。同时，为更好地适应无人机运动特征，在滤波中引入了轨迹置信度建模。通过结合运动特征提取模型和基于空间信息的WIoU匹配算法，设计了一种数据关联方法，综合目标的外观和运动信息以跟踪快速移动的目标。实验结果表明，WideTrack在VisDrone-MOT数据集上的MOTA分数较现有最优模型提高了5.3个百分点；该模型在VisDrone-MOT数据集和UAVDT数据集上的处理速度分别达到16?FPS和29?FPS，验证了其在无人机视频多目标跟踪任务中的有效性。

关键词: 多目标跟踪（MOT）, Transformer, 广视角目标捕捉, 轨迹置信度建模

Abstract: Multi-object tracking (MOT) in aerial drone videos presents significant challenges due to wide viewing angles, small and distant targets, and rapid target movements, which limit the effectiveness of traditional methods. This paper proposes WideTrack, an innovative Transformer-based MOT method, which employs a wide-angle feature fusion memory network to enhance tracking capability for small, distant targets. To adapt to drone motion, the paper integrates novel track confidence modeling into the filtering process. Additionally, it develops a data association strategy combining a motion feature extraction model and spatially informed WIoU matching, which effectively merges appearance and motion cues to track fast-moving targets. Experimental results on the VisDrone-MOT and UAVDT datasets demonstrate that WideTrack outperforms existing methods, establishing its efficacy and robustness in drone-based MOT tasks. Experimental results show that WideTrack improves the MOTA score by 5.3 percentage points over the best existing model on the VisDrone-MOT dataset. Moreover, the model achieves processing speeds of 16 frames per second on the VisDrone-MOT dataset and 29?frames per second on the UAVDT dataset, demonstrating its effectiveness in drone-based multi-object tracking tasks.

Key words: multi-object tracking(MOT), Transformer, wide-angle target acquisition, tracklet confidence modeling

张贝宁, 汤敏, 李洪均, 谢正光. 广视角特征融合记忆网络的多目标跟踪算法[J]. 计算机工程与应用, 2025, 61(23): 173-180.

ZHANG Beining, TANG Min, LI Hongjun, XIE Zhengguang. Multi-Object Tracking Algorithm with Wide-Angle Feature Fusion Memory Network[J]. Computer Engineering and Applications, 2025, 61(23): 173-180.

参考文献

[1] ALAVI A H, JIAO P C, BUTTLAR W G, et al. Internet of Things-enabled smart cities: state-of-the-art and future trends[J]. Measurement, 2018, 129: 589-606.
[2] YAACOUB J P, NOURA H, SALMAN O, et al. Security analysis of drones systems: attacks, limitations, and recommendations[J]. Internet of Things, 2020, 11: 100218.
[3] REZAEE K, ZADEH H G, CHAKRABORTY C, et al. Smart visual sensing for overcrowding in COVID-19 infected cities using modified deep transfer learning[J]. IEEE Transactions on Industrial Informatics, 2023, 19(1): 813-820.
[4] LIU S, LI X, LU H C, et al. Multi-object tracking meets moving UAV[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 8866-8875.
[5] YAO M F, WANG J Q, PENG J L, et al. FOLT: fast multiple object tracking from UAV-captured videos based on optical flow[C]//Proceedings of the 31st ACM International Conference on Multimedia. New York: ACM, 2023: 3375-3383.
[6] ZHANG Y F, WANG C Y, WANG X G, et al. FairMOT: on the fairness of detection and re-identification in multiple object tracking[J]. International Journal of Computer Vision, 2021, 129(11): 3069-3087.
[7] DENG C W, WU J P, HAN Y Q, et al. Learning a robust topological relationship for online multiobject tracking in UAV scenarios[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-15.
[8] ZENG F G, DONG B, ZHANG Y A, et al. MOTR: end-to-end multiple?object tracking with Transformer[C]//Proceedings of the European Conference on Computer Vision.Cham: Springer Nature Switzerland, 2022: 659-675.
[9] VASWANI A. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6000-6010.
[10] ALEXEY D, LUCAS?B, ALEXANDER?K, et al. An image is worth 16×16 words: Transformers for image recognition at scale[C]//Proceedings of the International Conference on Learning Representations, 2020:1-21.
[11] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with Transformers[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer International Publishing, 2020: 213-229.
[12] LIU Z, LIN Y T, CAO Y, et al. Swin Transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 9992-10002.
[13] BEWLEY A, GE Z Y, OTT L, et al. Simple online and realtime tracking[C]//Proceedings of the IEEE International Conference on Image Processing. Piscataway: IEEE, 2016: 3464-3468.
[14] CHEN L, AI H Z, ZHUANG Z J, et al. Real-time multiple people tracking with deeply learned candidate selection and person re-identification[C]//Proceedings of the IEEE International Conference on Multimedia and Expo. Piscataway: IEEE, 2018: 1-6.
[15] ZHANG Y F, SUN P Z, JIANG Y, et al. ByteTrack: multi-object tracking by associating every detection box[C]//Proceedings of the European Conference on Computer Vision.Cham: Springer Nature Switzerland, 2022: 1-21.
[16] SHUAI B, BERNESHAWI A, LI X Y, et al. SiamMOT: siamese multi-object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 12367-12377.
[17] MEINHARDT T, KIRILLOV A, LEAL-TAIXE L, et al. TrackFormer: multi-object tracking with transformers[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 8834-8844.
[18] MAGGIOLINO G, AHMAD A, CAO J K, et al. Deep OC-sort:multi-pedestrian tracking by adaptive re-identification[C]//Proceedings of the IEEE International Conference on Image Processing. Piscataway: IEEE, 2023: 3025-3029.
[19] HYUN J, KANG M, WEE D, et al. Detection recovery in online multi-object tracking with sparse graph tracker[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2023: 4839-4848.
[20] DU Y H, ZHAO Z C, SONG Y, et al. StrongSORT: make DeepSORT great again[J]. IEEE Transactions on Multimedia, 2023, 25: 8725-8737.
[21] YI K F, LUO K, LUO X L, et al. UCMCTrack: multi-object tracking with uniform camera motion compensation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2024: 6702-6710.