Multi-Lens Spectral Image Mosaic Method of Cultural Relics with Improved Grid Projection Transformation

doi:10.3778/j.issn.1002-8331.2310-0386

Abstract

Abstract: In order to solve the problem of the accumulated error and the relative position deviation caused by multiple depth of field parallax in the Mosaic of multi-lens spectral image sequences of cultural relics, an improved grid homologous projection transformation method is proposed. Firstly, the orthophoto color panoramic image is introduced as the reference image to avoid the cumulative error problem based on the single lens spectral image. The SIFT-LBP feature matching algorithm is proposed, and LBP texture is used to screen and supplement SIFT feature matching, and accurate feature matching point pairs are obtained. Then the grid homologous transformation is used to calculate the offset position of the pixel points in the target grid with the distance between the feature matching points as the weight to reduce the feature relative position offset problem. Finally, the histogram matching weighted fusion method is used to fuse the spectral image of the shooting lens to smooth the overlapping area of the image. Compared with the mainstream image splicing methods, this method achieves better visual effect, and PSNR and SSIM are increased by 12.81% and 13.12% respectively, providing reliable panoramic spectral image data for cultural relics protection and research.

Key words: image processing, image stitching, homologous projection transformation, spectral images of cultural relics, feature detection, image alignment

摘要： 针对文物多镜头光谱图像序列拼接时的累计误差和多个景深视差导致的特征相对位置偏移问题，提出改进网格单应性投影变换的拼接方法。引入正射彩色全景图像作为参考图像，避免了以单一分镜头光谱图像为基准的累计误差问题。提出SIFT-LBP特征匹配算法，利用LBP纹理对SIFT特征匹配进行筛选和补充，获得准确的特征匹配点对；采用网格单应性变换，以特征匹配点之间的距离为权重，计算目标网格像素点的偏移位置，减少特征相对位置偏移问题；采用直方图匹配加权融合方法融合分镜头光谱图像，以平滑图像重叠区域。与主流图像拼接方法相比较，该方法拼接的图像视觉效果更优，且峰值信噪比较主流方法提高12.81%，结构相似性提高13.12%，为文物保护与研究提供可靠的全景光谱图像数据。

关键词: 图像处理, 图像拼接, 单应性投影变换, 文物光谱图像, 特征检测, 图像配准

ZHENG Yangyang, WANG Huiqin, WANG Ke, WANG Zhan, ZHEN Gang, LI Yuan. Multi-Lens Spectral Image Mosaic Method of Cultural Relics with Improved Grid Projection Transformation[J]. Computer Engineering and Applications, 2024, 60(21): 225-235.

郑阳阳, 王慧琴, 王可, 王展, 甄刚, 李源. 改进网格单应性投影变换的文物多镜头光谱图像拼接方法[J]. 计算机工程与应用, 2024, 60(21): 225-235.

References

[1] CHEN G, LU H, ZOU W, et al. Spatiotemporal fusion for spectral remote sensing: a statistical analysis and review[J]. Journal of King Saud University-Computer and Information Sciences, 2023, 35(3): 259-273.
[2] 魏萍. 数字技术在文物保护中的运用及传播研究[J].中国民族博览, 2022(14): 199-202.
WEI P. Research on the application and dissemination of digital technology in cultural relics protection[J]. China National Exhibition, 2022(14): 199-202.
[3] YUSOF A A, YAHYA M F, MOHAMED NOR M K, et al. SUAS-based NDVI and RGB image for remote landscape and environmental monitoring on university campus[C]//Proceedings of the Advances in Intelligent Manufacturing and Mechatronics: Selected Articles from the Innovative Manufacturing, Mechatronics & Materials Forum (iM3F 2022), Pahang, Malaysia, 2023.
[4] 陈冬梅, 马亮亮, 张献明. 无损光谱技术在文物保护中的应用进展[J]. 光谱学与光谱分析, 2023, 43(2): 334-341.
CHEN D M, MA L L, ZHANG X M. Application progress of non-destructive spectroscopy on conversation of cultural relics[J].Spectroscopy and Spectral Analysis, 2023, 43(2): 334-341.
[5] SZELISKI R. Image alignment and stitching: a tutorial[J]. Foundations and Trends? in Computer Graphics and Vision, 2006, 2(1): 1-104.
[6] SAFFARI A, ZAHIRI S H, KHISHE M. Automatic recognition of sonar targets using feature selection in micro-doppler signature[J]. Defence Technology, 2023, 20: 58-71.
[7] CUI J, LIU M, ZHANG Z, et al. Robust UAV thermal infrared remote sensing images stitching via overlap-prior-based global similarity prior model[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 14: 270-282.
[8] YAN Y, XU L, LIU Y, et al. Larynx ultrasound image stitching based on multiconstraint super?pixel feature[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-11.
[9] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60: 91-110.
[10] BAY H, TUYTELAARS T, VAN GOOL L. SURF: speeded up robust features[C]//Proceedings of the European Conference on Computer Vision, 2006: 404-417.
[11] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proceedings of the 2011 International Conference on Computer Vision, 2011.
[12] 李焱磊, 刘静博, 刘文成, 等. 一种用于多视角雷达图像配准的改进RIFT算法[J].信号处理, 2023, 39(9): 1633-1650.
LI Y L, LIU J B, LIU W C, et al. An improved RIFT algorithm for multi-view radar image registration[J]. Journal of Signal Processing, 2023, 39(9): 1633-1650.
[13] 徐海青, 汪舒, 陈是同. 基于图像特征提取的红外影像动态配准方法研究[J]. 微型电脑应用, 2023, 39(5): 73-76.
XU H Q, WANG S, CHEN S T. Research on dynamic registration method of infrared image based on image feature extraction[J]. Microcomputer Applications, 2023, 39(5): 73-76.
[14] ZHANG Y, WAN Z, JIANG X, et al. Automatic stitching for hyperspectral images using robust feature matching and elastic warp[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 3145-3154.
[15] LIN C C, PANKANTI S U, NATESAN RAMAMURTHY K, et al. Adaptive as-natural-as-possible image stitching[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
[16] 贾停停, 王慧琴, 王可, 等. 相位相关性增强的自适应低重叠率多光谱图像快速拼接算法[J]. 液晶与显示, 2022, 37(4): 483-493.
JIA T T, WANG H Q, WANG K, et al. Adaptive low overlap multispectral image fast mosaic algorithm based on phasecorrelation enhancement[J]. Chinese Journal of Liquid Crystals and Displays, 2022, 37(4): 483-493.
[17] 李新胜, 刘宇. 基于边缘特征的虚实图像精确配准[J]. 四川大学学报(工程科学版), 2014, 46(5): 96-103.
LI X S, LIU Y. Precise registration of virtual and real image by edge feature[J]. Journal of Sichuan University (Engineering Science Edition), 2014, 46(5): 96-103.
[18] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999.
[19] ZARAGOZA J, CHIN T J, BROWN M S, et al. As-projective-as?possible image stitching with moving DLT[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013.
[20] BROWN M, LOWE D G. Automatic panoramic image stitching using invariant features[J]. International Journal of Computer Vision, 2007, 74: 59-73.
[21] GAO J, KIM S J, BROWN M S. Constructing image panoramas using dual?homography warping[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2011), 2011.
[22] LIN W Y, LIU S, MATSUSHITA Y, et al. Smoothly varying affine stitching[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2011), 2011.
[23] LI J, WANG Z, LAI S, et al. Parallax-tolerant image stitching based on robust elastic warping[J]. IEEE Transactions on Multimedia, 2017, 20(7): 1672-1687.
[24] CHANG C H, SATO Y, CHUANG Y Y. Shape-preserving half-projective warps for image stitching[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
[25] LEE K Y, SIM J Y. Warping residual based image stitching for large parallax[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[26] JIA Q, LI Z, FAN X, et al. Leveraging line-point consistence to preserve structures for wide parallax image stitching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021.
[27] MUJA M, LOWE D G. Fast approximate nearest neighbors with automatic algorithm configuration[C]//Proceedings of the Fourth International Conference on Computer Vision Theory and Applications, Lisboa, Portugal, February 5-8, 2009.
[28] ZHANG Z. Parameter estimation techniques: a tutorial with application to conic fitting[J]. Image and Vision Computing, 1997, 15(1): 59-76.
[29] LU F, HUANG J. An improved local binary pattern operator for texture classification[C]//Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016
[30] MA W, WANG K, LI J, et al. Infrared and visible image fusion technology and application: a review[J]. Sensors, 2023, 23(2): 599.
[31] LIAO T, LI N. Single-perspective warps in natural image stitching[J]. IEEE Transactions on Image Processing, 2019, 29: 724-735.
[32] MIKOLAJCZYK K, SCHMID C. A performance evaluation of local descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(10): 1615-1630.
[33] DU P, NING J, CUI J, et al. Geometric structure preserving warp for natural image stitching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.