基于特征增强的光学遥感视频多目标跟踪算法

doi:10.3778/j.issn.1002-8331.2309-0363

摘要/Abstract

摘要： 高分辨率视频卫星的出现为遥感视频目标的检测和跟踪提供了重要数据源，在灾害检测、军事侦察等领域都有重要作用。针对遥感视频复杂地物背景下目标尺寸弱小、目标特征相似、云雾遮挡和背景噪声干扰导致目标误检、误跟、漏跟问题，提出基于特征增强的光学遥感视频多目标跟踪算法FE-MOT，设计了基于边缘特征增强的特征提取网络，在微小目标缺乏纹理特征的情况下更好地融合语义与空间特征，通过构建基于交叉熵损失和中心损失的Re-ID分支结构，提高了对具有相似特征的微小目标的可分性。在吉林一号遥感视频多目标跟踪数据集AIR-MOT上的实验结果表明，FE-MOT在原模型的基础上MOTA提高了14.28个百分点，IDF1提高了15.47个百分点，FN降低了24个百分点，对于遥感视频多目标跟踪中目标身份维持能力和跟踪稳定性有显著提升，在2个Tesla T4 GPU上的运行速度达到19.9?FPS，满足实时运行的需要。

关键词: 视频卫星, 多目标跟踪, 联合检测跟踪

Abstract: The emergence of high-resolution video satellites provides an important data source for the detection and tracking of remote sensing video targets, and plays an important role in disaster detection, military reconnaissance and other fields. Aiming at the problems of false detection, false tracking and missed tracking caused by small target size, similarity of the targets, cloud obstruction, and background noise interference in remote sensing videos with complex ground objects, a multi-target tracking algorithm named FE-MOT for optical remote sensing videos based on feature enhancement is proposed. The feature extraction network based on edge feature enhancement is designed to efficiently integrate both semantic and spatial features under the lack of texture features of small targets. By constructing a Re-ID branch structure based on cross-entropy loss and center loss, it improves the separability of small targets with similar characteristics. Experimental results on the Jilin-1 remote sensing video multi-target tracking dataset AIR-MOT show that FE-MOT improves MOTA, IDF1 and FN by 14.28, 15.47 and 24?percentage points respectively compared with the original model, which helps to improve the target identity maintenance capability and tracking stability of the multiple targets in remote sensing videos significantly. And the running speed on two Tesla T4 GPUs reaches 19.9?FPS, meeting the requirements for real-time operation.

Key words: satellite video, multi-object tracking, joint detection and tracking

刘晨, 陈实, 陈红珍. 基于特征增强的光学遥感视频多目标跟踪算法[J]. 计算机工程与应用, 2025, 61(3): 177-185.

LIU Chen, CHEN Shi, CHEN Hongzhen. Multi-Object Tracking Algorithm Based on Feature Enhancement in Optical Remote Sensing Videos[J]. Computer Engineering and Applications, 2025, 61(3): 177-185.

参考文献

[1] FENG J, ZENG D, JIA X, et al. Cross-frame keypoint-based and spatial motion information-guided networks for moving vehicle detection and tracking in satellite videos[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 177: 116-130.
[2] 袁益琴, 何国金, 江威, 等. 遥感视频卫星应用展望[J]. 自然资源遥感, 2018, 30(3): 1-8.
YUAN Y Q, HE G J, JIANG W, et al. Application of earth observation system of video satellite[J]. Remote Sensing for Natural Resources, 2018, 30(3): 1-8.
[3] LUO W, XING J, MILAN A, et al. Multiple object tracking: a literature review[J]. Artificial Intelligence, 2021, 293: 103448.
[4] BEWLEY A, GE Z, OTT L, et al. Simple online and realtime tracking[C]//Proceedings of the IEEE 2016 IEEE International Conference on Image Processing (ICIP), 2016: 3464-3468.
[5] WOJKE N, BEWLEY A, PAULUS D. Simple online and realtime tracking with a deep association metric[C]//Proceedings of the IEEE 2017 IEEE International Conference on Image Processing (ICIP), 2017: 3645-3649.
[6] ZHU J, YANG H, LIU N, et al. Online multi-object tracking with dual matching attention networks[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 366-382.
[7] DAI P, WENG R, CHOI W, et al. Learning a proposal classifier for multiple object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 2443-2452.
[8] HE J, HUANG Z, WANG N, et al. Learnable graph matching: incorporating graph partitioning with deep feature learning for multiple object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 5299-5309.
[9] 袁德平, 史浩山, 郑娟毅. 用于多目标数据关联的群智能混合算法[J]. 华南理工大学学报 (自然科学版), 2012, 40(9): 97-103.
YUAN D P, SHI H S, ZHENG J Y. Hybrid swarm intelligent algorithm for multi-target data association[J]. Journal of South China University of Technology (Natural Science Edition), 2012, 40(9): 97-103.
[10] YANG F, ODASHIMA S, MASUI S, et al. Hard to track objects with irregular motions and similar appearances? make it easier by buffering the matching space[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023: 4799-4808.
[11] QIN Z, ZHOU S, WANG L, et al. MotionTrack: learning robust short-term and long-term motions for multi-object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023: 17939-17948.
[12] BERGMANN P, MEINHARDT T, LEAL-TAIXE L. Tracking without bells and whistles[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019: 941-951.
[13] WANG Z, ZHENG L, LIU Y, et al. Towards real-time multi-object tracking[C]//Proceedings of the 16th European Conference on Computer Vision (ECCV 2020), Glasgow, UK, 2020: 107-122.
[14] ZHOU X, KOLTUN V, KRAHENBUHL P. Tracking objects as points[C]//Proceedings of the 16th European Conference on Computer Vision (ECCV 2020), Glasgow, UK, 2020: 474-490.
[15] ZHANG Y, WANG C, WANG X, et al. FairMOT: on the fairness of detection and Re-IDentification in multiple object tracking[J]. International Journal of Computer Vision, 2021, 129: 3069-3087.
[16] ZHANG Y, SUN P, JIANG Y, et al. ByteTrack: multi-object tracking by associating every detection box[C]//Proceedings of the European Conference on Computer Vision, 2022: 1-21.
[17] WANG Y, KITANI K, WENG X. Joint object detection and multi-object tracking with graph neural networks[C]//Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021: 13708-13715.
[18] WU J, SU X, YUAN Q, et al. Multivehicle object tracking in satellite video enhanced by slow features and motion features[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-26.
[19] YIN Q, HU Q, LIU H, et al. Detecting and tracking small and dense moving objects in satellite videos: a benchmark[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-18.
[20] HE Q, SUN X, YAN Z, et al. Multi-object tracking in satellite videos with graph-based multitask modeling[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-13.
[21] XIAO C, WU S, WANG Y, et al. RSMOT: remote sensing multi-object tracking network with local motion prior for objects in satellite videos[C]//Proceedings of the 2022 IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2022), 2022: 1904-1907.
[22] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017: 136-144.
[23] ZHANG T, CHEN H, CHEN S, et al. Edge-enhanced efficient network for remote sensing image super-resolution[J]. International Journal of Remote Sensing, 2022, 43(14): 5324-5347.
[24] YU F, WANG D, SHEHHAMER E, et al. Deep layer aggregation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 2403-2412.
[25] 贾天豪, 彭力, 戴菲菲. 引入残差学习与多尺度特征增强的目标检测器[J]. 计算机科学与探索, 2023, 17(5): 1102-1111.
JIA T H, PENG L, DAI F F. Object detector with residual learning and multi-scale feature enhancement[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(5): 1102-1111.
[26] WEN Y, ZHANG K, LI Z, et al. A discriminative feature learning approach for deep face recognition[C]//Proceedings of the 14th European Conference on Computer Vision (ECCV 2016), Amsterdam, The Netherlands, 2016: 499-515.
[27] MUNKRES J. Algorithms for the assignment and transportation problems[J]. Journal of the Society for Industrial and Applied Mathematics, 1957, 5(1): 32-38.
[28] BERNARDIN K, STIEFELHAGEN R. Evaluating multiple object tracking performance: the clear mot metrics[J]. EURASIP Journal on Image and Video Processing, 2008: 246309.