Research on UAV Image Stitching Technology Based on IB-SURF Algorithm

doi:10.3778/j.issn.1002-8331.2209-0151

Abstract

Abstract: Aiming at the high computational complexity and low feature matching rate of the traditional SURF algorithm (speeded up robust features), when stitching multiple, high-resolution UAV images, the UAV images stitching technology based on the IB-SURF algorithm is proposed. Firstly, the overlapping area of the image is obtained by combining the POS (position and orientation system) information of the UAV. Then, a mask is constructed to detect feature points in the overlapping area of UAV images to reduce the time of feature extraction. Using the idea of Image Block (IB), the image is divided into grids, and the feature points are simply screened. Finally, neighborhood-K nearest neighbors (neighborhood-KNN) is introduced to match feature points to improve the efficiency of image matching. Experimental results show that IB-SURF algorithm has faster running speed and higher feature matching rate, the average feature matching rate reaches 84.3%, and the feature matching accuracy exceeds 95.1%. It provides a technical basis for high quality image stitching.

Key words: UAV images, image block-speeded up robust features (IB-SURF) algorithm, feature point extraction, image block

摘要： 针对传统SURF算法（speeded up robust features）在拼接高分辨率无人机航拍图像时运行速度慢、特征匹配率低的特点，提出了一种基于IB-SURF（image block-SURF）技术的无人机图像拼接算法。结合无人机定位定姿系统（position and orientation system，POS）求取图像重叠区域；构造掩模在无人机图像重叠区域检测特征点，减少特征提取时间；借助图像分块（image block，IB）的思想对图像划分网格，精简筛选特征点；引入Neighborhood-KNN （neighborhood-K nearest neighbors）进行特征点匹配，提高图像匹配效率。实验结果表明，IB-SURF算法有较快的运行速度和较高的特征匹配率，平均特征匹配率达到84.3%，特征匹配正确率超过95.1%，为图像高质量拼接提供了技术基础。

关键词: 无人机图像, IB-SURF算法, 特征点提取, 图像分块

JIANG Zhi, JIANG Degang, HUANG Zijie, GUO Cailing, LI Bailin. Research on UAV Image Stitching Technology Based on IB-SURF Algorithm[J]. Computer Engineering and Applications, 2024, 60(3): 263-269.

江智, 江德港, 黄子杰, 郭彩玲, 李柏林. 基于IB-SURF算法的无人机图像拼接技术研究[J]. 计算机工程与应用, 2024, 60(3): 263-269.

References

[1] ZHANG W, LI X, YU J, et al. Remote sensing image mosaic technology based on SURF algorithm in agriculture[J]. EURASIP Journal on Image Video Processing, 2018, 85.
[2] LI Y S, PEI J H, XIE W X, et al. A new fast automatic mosaic method on unmanned aerial vehicle images[J]. Acta Electronica Sinica, 2012, 40(5): 935-940.
[3] TSAI C H, LIN Y C. An accelerated image matching technique for UAV orthoimage registration[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 128: 130-145.
[4] WANG B, WANG W F. Unmanned aerial vehicle image mosaic algorithm based on local gray fitting[J]. Journal of China University of Petroleum, 2009, 33(2): 169-173.
[5] WANG Y Z, ZHAO D Q, ZHANG N X, et al. Stitching method based on drone POS information[J]. Computer System Applications, 2018, 27(3): 112-117.
[6] WANG Z C, CHEN Y F, ZHU Z W, et al. An automatic panoramic image method based on graph model[J]. Multimedia Tools Application, 2016, 75(5): 2725-2740.
[7] 向程谕, 王冬丽, 李建勋, 等. 基于改进 SIFT特征的深度图像匹配[J]. 计算机应用, 2016, 36(S2): 135-138.
XIANG C Y, WANG D L, LI J X, et al. Deep image matching based on improved SIFT features[J]. Journal of Computer Applications, 2016, 36(S2): 135-138.
[8] 权巍, 包铁壮, 白宝兴, 等. SURF与RANSAC 算法结合的图像跟踪方法[J]. 计算机仿真, 2016, 33(9): 268-272.
QUAN W, BAO T Z, BAI B X, et al. A method of real-time picture tracking and mapping based on SURF algorithm and RANSAC algorithm[J]. Computer Simulation, 2016, 33(9): 268-272.
[9] GUI Q Y, CHANG X, JIANG Z. A fast aerial images mosaic method based on ORB feature and homography matrix[C]// Proceedings of the 2019 International Conference on Computer, Information and Telecommunication Systems (CITS), Beijing, China, Aug 28-31, 2019: 1-5.
[10] CUI H, LI Y, ZHANG K. A fast UAV aerial image mosaic method based on improved KAZE[C]//Proceedings of the 2019 Chinese Automation Congress (CAC), Hangzhou, China, Nov 22-24, 2019: 2427-2432.
[11] LI J Y, WANG Y L , WANG Y J. Visual tracking and learning using speeded up robust features[J]. Pattern Recognition Letters, 2012, 33(16): 2094-2101.
[12] 罗久飞, 邱广, 张毅, 等.基于自适应双阈值的 SURF 双目视觉匹配算法研究[J]. 仪器仪表学报, 2020, 41(3): 240-247.
LUO J F, QIU G, ZHANG Y, et al. Research on speeded up robust feature binocular vision matching algorithm based on adaptive double threshold[J]. Chinese Journal of Scientific Instrument, 2020, 41(3): 240-247.
[13] 丁小奇, 李健, 胡雅婷, 等. 基于改进SURF算法无人机影像特征匹配的研究[J]. 中国农机化学报, 2020, 41(2): 147-154.
DING X Q, LI J, HU Y T, et al. Research on UAV image feature matching based on improved SURF algorithm[J]. Journal of Chinese Agricultural Mechanization, 2020, 41(2): 147-154.
[14] 潘梅, 李磊民.基于改进Harris-SURF的无人机图像快速拼接[J].微型机与应用, 2014, 33(15): 43-46.
PAN M, LI L M. Fast UAV image mosaic based on improved Harris-SURF[J]. Microcomputer & its Applications, 2014, 33(15): 43-46.
[15] HU M T, PENG Y , XU Y, Fast image matching algorithm based on improved SURF[J]. Transducer?and?Microsystem Technology, 2017, 36(11): 151-153.
[16] SHAO R Z, DU C, CHEN H et al, Fast anchor point matching for emergency UAV image stitching using position and pose information[J]. Sensors, 2020, 20(7): 2007-2024.
[17] AN Q, CHEN X J, WU S S. A novel fast image stitching method based on the combination of SURF and cell[J]. Complexity, 2021, 9995030.
[18] 孙灏, 高俊强, 许苏苏. 基于SURF算法和改进RANSAC算法的无人机影像匹配[J]. 测绘工程, 2017, 26(1): 55-59.
SUN H, GAO J Q, XU S S. UAV image matching based on SURF algorithm and improved RANSAC algorithm[J]. Engineering of Surveying and Mapping, 2017, 26(1): 55-59.
[19] 史露, 苏刚, 韩飞. 基于SIFT和SURF图像拼接算法的改进算法[J]. 计算机应用与软件, 2013(6): 72-74.
SHI L, SU G, HAN F. An improved algorithm based on SIFT and SURF image stitching algorithm[J]. Computer Applications and Software, 2013(6): 72-74.
[20] 魏利胜, 甘泉. 基于小波变换的新型SURF图像拼接方法[J]. 电子测量与仪器学报, 2017, 31(5): 766-772.
WEI L S, GAN Q. Novel SURF image mosaic method based on wavelet transform[J].Journal of Electronic Measurement and Instrumentation, 2017, 31(5): 766-772.
[21] 韩敏, 闫阔, 秦国帅. 基于改进KAZE的无人机航拍图像拼接算法[J]. 自动化学报, 2019, 45(2): 305-314.
HAN M, YAN K, QIN G S.A mosaic algorithm for UAV aerial image with improved KAZE[J].Acta Automatica Sinica, 2019, 45(2): 305-314.
[22] BIAN J, LIN W Y, MATSUSHITA Y, et al. GMS: grid-based motion statistics for fast, ultra-robust feature correspondence[C]//Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition, 2017.
[23] ZHANG F, LIU F. Parallax-tolerant image stitching[C]//IEEE Conference on Computer Vision & Pattern Recognition, 2014.
[24] CHEN W, LIU Y, WANG Y W et al, Fast image stitching algorithm based on improved FAST-SURF[J]. Applied Optics, 2021, 42(4): 636-642.
[25] WANG R, SHI Y J , CAO W M. GA-SURF: a new speeded-up robust feature extraction algorithm for multispectral images based on geometric algebra[J]. Pattern Recognition Letters, 2019, 127: 11-17.
[26] 刘婷婷, 张惊雷.基于ORB特征的无人机遥感图像拼接改进算法[J]. 计算机工程与应用, 2018, 54(2): 193-197.
LIU T T, ZHANG J L. Improved image stitching algorithm based on ORB features by UAV remote sensing[J]. Computer Engineering and Applications, 2018, 54(2): 193-197.