Real-Time Image Stabilization Algorithm for Airborne Video

doi:10.3778/j.issn.1002-8331.2103-0264

Abstract

Abstract: To solve the problem of image instability in real-time image stabilization algorithm due to the sparse background and rapid movement of foreground in airborne video intake and monitoring, an adaptive Shi-Tomasi real-time optimization image stabilization algorithm for airborne target in airborne video is proposed. Firstly, the Shi-Tomasi corner detection threshold is adaptively changed according to the distribution of extracted feature points to solve the problem that a single threshold cannot adapt to the extraction of feature points from complex and sparse background in the air. Then a real-time optimization algorithm with constraints is constructed to calculate a smooth video path and deal with excessive image offset caused by the lack of constraints in the filtering algorithm. The comparative experimental results show that the adaptive Shi-Tomasi optimized image stabilization algorithm can cope with the image stabilization of various airborne sparse background videos, and solve the problem of large image offset after image stabilization of fast-moving targets. With improved stability, the speed can reach more than 20 frame/s, which meets the real-time processing requirements.

Key words: airborne video, complex sparse background, moving object, adaptive corner detection, real time optimization

摘要： 针对机载飞机视频摄取与监视中，由于背景稀疏和前景的大幅度快速运动，造成实时稳像算法存在的画面不稳定的问题，提出了自适应Shi-Tomasi机载视频空中目标实时优化稳像算法。根据提取特征点分布自适应地改变Shi-Tomasi角点检测阈值，解决单一阈值不能适应空中复杂稀疏背景特征点提取的问题。构建带约束的实时优化算法，计算平滑的视频路径，解决基于滤波的算法缺少约束导致画面偏移过大的问题。比对实验结果表明，自适应Shi-Tomasi优化稳像算法能够应对各类机载稀疏背景视频的稳像，解决了快速运动目标稳像后画面大幅偏移的问题，稳定性提高，速度达到20?frame/s以上，满足实时处理需求。

关键词: 机载视频, 复杂稀疏背景, 运动目标, 自适应角点检测, 实时优化

LIU Yuqing, LIU Cheng, AO Xuecong. Real-Time Image Stabilization Algorithm for Airborne Video[J]. Computer Engineering and Applications, 2022, 58(20): 212-219.

刘玉清, 刘成, 敖雪聪. 机载视频空中目标实时优化稳像算法[J]. 计算机工程与应用, 2022, 58(20): 212-219.

References

[1] 魏闪闪，谢巍，贺志强.数字视频稳像技术综述[J].计算机研究与发展，2017，54（9）：2044-2058.
WEI Shanshan，XIE Wei，HE Zhiqiang.Digital video stabilization techniques：a survey[J].Journal of Computer Research and Development，2017，54（9）：2044-2058.
[2] 尹丽华.基于车载运动平台的全景稳像关键技术研究[D].上海：中国科学院大学（中国科学院上海技术物理研究所），2018.
YIN Lihua.Study on key technology of panoramic image stabilization based on vehicle-mounted platform[D].Shanghai：University of Chinese Academy of Sciences（Shanghai Institute of Technical Physics，Chinese Academy of Sciences），2018.
[3] 钟平.机载电子稳像技术研究[D].长春：中国科学院研究生院（长春光学精密机械与物理研究所），2003.
ZHONG Ping.Study on electronic image stabilization technology for the image sequences of aero-borne camera system[D].Changchun：University of Chinese Academy of Sciences（Changchun Institute of Optics，Fine Mechanics and Physics，Chinese Academy of Sciences），2003.
[4] 杨毅.电子稳像算法研究与实现[D].南京：南京理工大学，2009.
YANG Yi.Research and implementation of electronic image stabilization algorithm[D].Nanjing：Nanjing University of Science and Technology，2009.
[5] 苑爱博.机载电子稳像系统设计与实现[D].长沙：国防科学技术大学，2010.
YUAN Aibo.Design and implementation of system for air-borne electronic image stabilization[D].Changsha：National University of Defense Technology，2010.
[6] 李迪，陈向坚，续志军，等.模糊神经网络在机载相机稳像中的应用[J].浙江大学学报（工学版），2012，46（8）：1540-1545.
LI Di，CHEN Xiangjian，XU Zhijun，et al.Type-II fuzzy neural networks with self-organizing recurrent intervals for image stabilization of the airborne camera[J].Journal of Zhejiang University（Engineering Science），2012，46（8）：1540-1545.
[7] 酒锐波，李冰寒，刘玉婷.基于FAST-9角点与光流法的机载视频稳像算法[J].动力与电气工程，2019，17（1）：38-42.
JIU Ruibo，LI Binghan，LIU Yuting.Airborne video image stabilization algorithm based on FAST-9 corner and optical flow method[J].Science & Technology Information，2019，17（1）：38-42.
[8] 王施鳗，许文海，董丽丽，等.基于改进的Harris角点的机载红外图像电子稳像[J].红外技术，2020，42（6）：573-579.
WANG Shiman，XU Wenhai，DONG Lili，et al.Electronic image stabilization of airborne infrared images based on improved Harris corner detection[J].Infrared Technology，2020，42（6）：573-579.
[9] 鲜勇，董力文.基于角点特征的机载摄像系统电子稳像算法[J].光学与光电技术，2016，14（2）：54-59.
XIAN Yong，DONG Liwen.An electronic stabilizing algorithm for airborne camera system[J].Optics & Optoelectronic Technology，2016，14（2）：54-59.
[10] 陈晓露.航空光电成像电子稳像技术研究[D].长春：中国科学院研究生院（长春光学精密机械与物理研究所），2014.
CHEN Xiaolu.Study on digital image stabilization technology for aerial opto-electric imaging system[D].Changchun：University of Chinese Academy of Sciences（Changchun Institute of Optics，Fine Mechanics and Physics，Chinese Academy of Sciences），2014.
[11] GRUNDMANN M，KWATRA V，ESSA I.Auto-directed video stabilization with robust L1 optimal camera paths[C]//2011 IEEE Conference on Computer Vision and Pattern Recognization（CVPR），2011：225-232.
[12] SHI J，TOMASI C.Good features to track[C]//IEEE Conference on Computer Vision and Pattern Recognition，1994.
[13] LUCAS B，KANADE T.An iterative image registration technique with an application to stereo vision[C]//Proc of Int Joint Conf on Artificial In telligence，1981：674-679.
[14] KALMAN R E.A new approach to linear filtering and prediction problems[J].Journal of Basic Engineering，1960，82（1）：35-45.
[15] STELLATO B，BANJAC G，GOULART P，et al.OSQP：an operator splitting solver for quadratic programs[J].Mathe-
matical Programming Computation，2020，12（4）：637-672.
[16] WANG M.Deep online video stabilization with multi-grid warping transformation learning[J].IEEE Transactions on Image Processing，2019，28（5）：2283-2292.