Research on Stereo Matching Algorithm Based on Multi-feature Fusion and Adaptive Aggregation

doi:10.3778/j.issn.1002-8331.2104-0353

Abstract

Abstract:

Aiming at the low matching accuracy of local stereo matching in illumination distortion and weak texture areas, a stereo matching algorithm based on cost calculation of multi-feature fusion and adaptive cross window aggregation is proposed. Firstly, the HSV color space component is introduced, combined with the improved Census transform and gradient information to calculate the matching cost, which eliminates the influence of the parallax boundary abnormal value, and enhances the algorithm’s robustness to illumination distortion. Secondly, an adaptive window cost aggregation method based on gradient information and variable color threshold is proposed to improve the matching accuracy of weak texture areas. Finally, the final disparity result is obtained by multi-step disparity refinement. Experimental results show that, compared with AD-Census algorithm, the mismatching rate of proposed algorithm is reduced by 3.24% under the same illumination exposure conditions. It can effectively solve the problem of mismatching parallax boundary and weak texture region, and has good robustness to illumination distortion and can effectively suppress noise interference.

Key words: machine vision, stereo matching, multi-feature fusion, adaptive window, radiometric distortion

摘要：

针对局部立体匹配在光照失真和弱纹理区域匹配精度低的问题，提出了一种多特征融合的代价计算和自适应十字窗口聚合的立体匹配算法。引入HSV颜色空间分量，结合改进后的Census变换和梯度信息作为匹配代价计算方法，排除了视差边界异常值的影响，增强了算法对光照失真的稳健性；提出了基于梯度信息和可变颜色阈值的自适应窗口代价聚合方法，提高了在弱纹理区域的匹配精度；通过视差计算和多步骤的视差精细得到了最终的视差结果。实验结果表明，所提算法较AD-Census算法在无光照失真条件下误匹配减少了3.24%，能有效解决视差边界和弱纹理区域错误匹配的问题，对光照失真稳健性好且能有效抑制噪声干扰。

关键词: 机器视觉, 立体匹配, 多特征融合, 自适应窗口, 光照失真

CHANG Yawen, ZHAO Dongqing, SHAN Yanhu. Research on Stereo Matching Algorithm Based on Multi-feature Fusion and Adaptive Aggregation[J]. Computer Engineering and Applications, 2021, 57(23): 219-225.

畅雅雯，赵冬青，单彦虎. 多特征融合和自适应聚合的立体匹配算法研究[J]. 计算机工程与应用, 2021, 57(23): 219-225.

References

[1] 陈炎，杨丽丽，王振鹏.双目视觉的匹配算法综述[J].图学学报，2020，41（5）：702-708.
CHEN Y，YANG L L，WANG Z P.Literature survey on stereo vision matching algorithms[J].Journal of Graphics，2020，41（5）：702-708.
[2] SCHARSTEIN D，SZELISKI R.A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J].International Journal of Computer Vision，2002，47：7-42.
[3] HIRSCHMULLER H.Stereo processing by semi-global matching and mutual information[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2007，30（2）：328-341.
[4] 郭鑫，王延杰，付东辉.三种测度融合的加权引导滤波匹配算法[J].计算机工程与科学，2019，41（6）：1044-1049.
GUO X，WANG Y J，FU D H.A weight guided filtering matching algorithm combining three measures[J].Computer Engineering & Science，2019，41（6）：1044-1049.
[5] JIAO J，WANG R，WANG W，et al.Local stereo matching with improved matching cost and disparity refinement[J].IEEE Multi-Media，2014，21（4）：16-27.
[6] MEI X，SUN X，ZHOU M C，et al.On building an accurate stereo matching system on graphics hardware[C]// 2013 IEEE International Conference on Computer Vision Workshops，2013：467-474.
[7] ZHANG K，FANG Y，MIN D，et al.Cross-scale cost aggregation for stereo matching[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition，2014：1590-1597.
[8] QU Y F，JIANG J X，DENG X J，et al.Robust local stereo matching under varying radiometric conditions[J].IET Computer Vision，2014，8（4）：263-276.
[9] HOSNI A，BLEYER M，GELAUTZ M.Secrets of adaptive support weight techniques for local stereo matching[J].Computer Vision and Image Understanding，2013，117（6）：620-632.
[10] 苏修，陈晓冬，徐怀远，等.基于HSV颜色空间的自适应窗口局部匹配算法[J].激光与光电子学进展，2018，55（3）：287-294.
SU X，CHEN X D，XU H Y，et al.Adaptive window local matching algorithm based on HSV color space[J].Laser & Optoelectronics Progress，2018，55（3）：287-294.
[11] 王云峰，吴炜，余小亮，等.基于自适应权重 AD-Census变换的双目立体匹配[J].工程科学与技术，2018，50（4）：153-160.
WANG Y F，WU W，YU X L，et al.A stereo matching system with the adaptive weight AD Census[J].Advanced Engineering Sciences，2018，50（4）：153-160.
[12] 欧永东，谢小鹏.基于改进Census变换的多特性立体匹配算法[J].计算机工程与科学，2020，42（6）：1030-1036.
OU Y D，XIE X F.A multi-feature stereo matching algorithm based on improved Census transform[J].Computer Engineering & Science，2020，42（6）：1030-1036.
[13] 程德强，李海翔，寇旗旗，等.融合边缘保持与改进代价聚合的立体匹配算法[J].中国图象图形报，2021，26（2）：438-451.
CHENG D Q，LI H X，KOU Q Q，et al.Stereo matching algorithm based on edge preservation and improved cost aggregation[J].Journal of Image and Graphics，2021，26（2）：438-451.
[14] WANG D，LIU H，CHENG X.A miniature binocular endoscope with local feature matching and stereo matching for 3D measurement and 3D reconstruction[J].Sensors，2018，18（7）：2243.
[15] 潘卫华，杜旭.基于改进自适应权重的三边滤波立体匹配算法[J].计算机工程与应用，2020，56（18）：186-192.
PAN W H，DU X.Three-edge filtering stereo matching algorithm based on improved adaptive support weight[J].Computer Engineering and Applications，2020，56（18）：186-192.
[16] 祝世平，李政.基于改进梯度和自适应窗口的立体匹配算法[J].光学学报，2015，35（1）：123-131.
ZHU S P，LI Z.A stereo matching algorithm using improved gradient and adaptive window[J].Acta Optica Sinica，2015，35（1）：123-131.
[17] YAO M，OUYANG W B，XU B G，et al.Hybrid cost aggregation for dense stereo matching[J].Multimedia Tools and Applications，2020，79（6）：23189-23202.
[18] HAMZAH R A，IBRAHIM H.Improvement of disparity map refinement stage using adaptive least square plane fitting technique[J].Electronics Letters，2020，56（18）：918-920.
[19] ZHU S P，YAN L N.Local stereo matching algorithm with efficient matching cost and adaptive guided image filter[J].Visual Computer，2017，33（9）：1087-1102.