Target Detecction and Real-Time Following for Unmanned Aerial Vehicle

doi:10.3778/j.issn.1002-8331.2302-0281

Abstract

Abstract: With the increasing application of drones in various fields, the demand for drone regulation is also gradually increasing. At the same time, due to the limited computing power and energy of drone platforms, effective detection and following algorithms are particularly important. Currently, deep learning-based methods are very effective for target detection, but there are still issues with stability, safety and interference from target shadows when directly applied to aerial target tracking tasks. To address the problem of shadow interference during target detection, a shadow recognition algorithm based on the HSV color space is proposed, which can segment and identify the shadow areas of the detected object, thus eliminating the interference of shadows on target detection. In order to obtain more accurate 3D position of the target drone, a new positioning algorithm is designed, which combines the center point of the detection box with the relatively fixed center point of the target drone through weighted fusion, reducing the impact of target box size fluctuations on target position estimation. In the obstacle avoidance strategy, drone-related constraints are integrated to avoid excessive oscillation during drone following. Additionally, a dynamic self-localization algorithm is used to detect and correct the control results of the tracked drone in real-time, improving the robustness of the following task. The proposed algorithm is validated through simulation on the Unreal Engine 4 platform and physical experiments, which can keep the accuracy of drone following tasks at a level of 0.1 meters.

Key words: unmanned aerial vehicle (UAV) following, target detection, shadow recognition, center positioning, dynamic self positioning

摘要： 随着无人机在各个领域的应用越来越广泛，目前对于无人机的管制需求也逐步上升，同时由于无人机平台算力、能源有限，有效的检测与跟随算法显得尤为重要。基于深度学习的方法对于目标检测十分有效，但其直接应用于空中目标跟随这一任务还存在稳定性与安全性不足以及目标阴影的干扰这些问题。针对目标检测时阴影干扰问题，提出了基于HSV色彩空间的阴影识别算法，能够对检测对象阴影区域进行分割识别，从而排除阴影对目标检测的干扰；为了得到了更精准的目标无人机三维位置，设计了二次定位算法，将纯检测框中心点与目标无人机结构上相对固定中心点进行了加权融合，减少了目标框大小浮动对目标位置估计的影响；在避障策略中融合了无人机相关约束以此避免了无人机跟随时的过度震荡，并利用动态环境下的自定位算法对追踪无人机的控制结果进行检测与实时修正，提升整个动态跟随过程中的鲁棒性。所提算法经虚幻4平台下的仿真与实物实验中得到验证，能够将无人机跟随任务的跟随精度控制在0.1 m级。

关键词: 无人机跟随, 目标检测, 阴影辨识, 中心定位, 动态自定位

LIU Rongqi, WANG Hongyu, HAN Jiaozhi. Target Detecction and Real-Time Following for Unmanned Aerial Vehicle[J]. Computer Engineering and Applications, 2024, 60(11): 319-327.

刘瑢琦, 王红雨, 韩佼志. 面向无人机目标的检测与实时跟随[J]. 计算机工程与应用, 2024, 60(11): 319-327.

References

[1] LIN F, PENG K, DONG X, et al. Vision-based formation for UAVs[C]//Proceedings of the 11th IEEE International Conference on Control & Automation, 2014: 1375-1380.
[2] AKER C, KALKAN S. Using deep networks for drone detection[C]//Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance, 2017: 1-6.
[3] WU M, XIE W, SHI X, et al. Real-time drone detection using deep learning approach[C]//Proceedings of the International Conference on Machine Learning and Intelligent Communications, 2018: 22-32.
[4] SHIH T K, TANG N C, HWANG J N. Exemplar-based video inpainting without ghost shadow artifacts by maintaining temporal continuity[J]. IEEE Transactions on Circuits & Systems for Video Technology, 2009, 19(3): 347-360.
[5] KHAN S H, BENNAMOUN M, SOHEL F, et al. Automatic shadow detection and removal from a single image[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 38(3): 431-446.
[6] QU L, TIAN J, HE S, et al. DeshadowNet: a multi-context embedding deep network for shadow removal[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4067-4075.
[7] WANG J, LI X, HUI L, et al. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018: 1788-1797.
[8] HU X, FU C W, LEI Z, et al. Direction-aware spatial context features for shadow detection and removal[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2020, 42(11): 2795-2808.
[9] ZHANG L, ZHANG Q, XIAO C. Shadow remover: image shadow removal based on illumination recovering optimization[J]. IEEE Transactions on Image Processing, 2015, 24(11): 4623-4636.
[10] INOUE N, YAMASAKI T. Learning from synthetic shadows for shadow detection and removal[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(11): 4187-4197.
[11] ODELGA M, STEGAGNO P, BüLTHOFF H H. Obstacle detection, tracking and avoidance for a teleoperated UAV[C]//Proceedings of the 2016 IEEE International Conference on Robotics and Automation, 2016: 2984-2990.
[12] WANG Y, JI J, WANG Q, et al. Autonomous flights in dynamic environments with onboard vision[C]//Proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2021: 1966-1973.
[13] JI J, PAN N, XU C, et al. Elastic tracker: a spatio-temporal trajectory planner flexible aerial tracking[C]//Proceedings of the 2022 International Conference on Robotics and Automation, 2022: 47-53.
[14] ZHOU X, YU X, PENG X. UAV collision avoidance based on varying cells strategy[J]. IEEE Transactions on Aerospace and Electronic Systems, 2019, 55(4): 1743-1755.
[15] HSU Y H, GAU R H. Reinforcement learning-based collision avoidance and optimal trajectory planning in UAV communication networks[J]. IEEE Transactions on Mobile Computing, 2022, 21(1): 306-320.
[16] YU X D, KUAN T W, QIAN Z Y, et al. HSV semantic segmentation on partially facility and phanerophyte sunshine-shadowing road[C]//Proceedings of the 2022 10th International Conference on Orange Technology, 2022: 1-4.
[17] KOUTSIOU D, SAVELONAS M, IAKOVIDIS D K. H-V shadow detection based on electromagnetism-like optimization[C]//Proceedings of the 2020 28th European Signal Processing Conference, 2021: 635-639.
[18] XU P Y, WU Y J, ZUO J X, et al. Longitudinal attitude control of UAV based on fuzzy PID[C]//Proceedings of the 2018 IEEE CSAA Guidance, Navigation and Control Conference, 2018: 1-5.
[19] HUANG T, HUANG D, LUO D. Attitude tracking for a quadrotor UAV based on fuzzy PID controller[C]//Proceedings of the 2018 5th International Conference on Information, Cybernetics, and Computational Social Systems, 2018: 1-6.
[20] SHENG G, GAO G. Research on the attitude control of civil quad-rotor UAV based on fuzzy PID control[C]//Proceedings of the 2019 Chinese Control and Decision Conference, 2019: 4566-4569.