融合频域信息的双目图像超分辨重建Mamba网络

doi:10.3778/j.issn.1002-8331.2503-0005

摘要/Abstract

摘要： 双目图像超分辨（stereo image super-resolution，SSR）旨在结合双目相机两个不同视角的图像信息，生成具有更高分辨率和更丰富细节的图像。当前，基于Transformer的SSR方法通过自注意力机制可以捕获全局的依赖关系，并获取广泛的上下文信息，但其计算复杂度随序列长度的增加呈平方级增长，运行效率不高。因此，提出基于Mamba架构的双目图像超分辨重建网络模型（MambaSSR）。Mamba架构具有线性计算复杂度，可以明显提高算法的运行效率，并具备和Transformer同样强大的功能。另外，在提出的MambaSSR模型中还设计了频域注意力模块，以保留关键的频率信息，从而更准确地重建图像的纹理细节。在多个公开数据集上进行了实验，结果表明，与主流方法相比，所提Mamba模型能够以更高的运行效率实现优越的超分辨重建效果。

关键词: 双目图像, 图像超分辨重建, Mamba, 频域

Abstract: Stereo image super-resolution (SSR) aims to exploit image information from two different perspectives of the binocular camera to generate images with higher resolution and richer details. Currently, Transformer-based SSR methods can capture global dependencies and obtain extensive contextual information through self-attention mechanisms. However, the computational complexity gets a quadratic increase with the sequence length, resulting in a low efficiency. Therefore, a new stereo image super-resolution network model based on Mamba architecture (MambaSSR) is proposed. The Mamba architecture has linear computational complexity, which can significantly improve the running efficiency of algorithms, and has the same powerful functions as Transformers. In addition, a frequency domain attention module is designed in the proposed MambaSSR model to preserve key frequency information. Thereby, more accurate texture details are reconstructed. Experiments are conducted on several public datasets, and the experimental results show that compared with mainstream methods, the proposed Mamba model can achieve excellent super-resolution reconstruction results with higher running efficiency.

Key words: stereo image, image super-resolution, Mamba, frequency domain

马鸣声, 张德. 融合频域信息的双目图像超分辨重建Mamba网络[J]. 计算机工程与应用, 2026, 62(8): 298-307.

MA Mingsheng, ZHANG De. Frequency-Assisted Mamba Network for Stereo Image Super-Resolution[J]. Computer Engineering and Applications, 2026, 62(8): 298-307.

参考文献

[1] 王应谦，王龙光，梁政宇，等.双目图像超分辨研究综述[J].激光与光电子学进展，2021，58（18）：306-320.
WANG Y Q，WANG L G，LIANG Z Y，et al.Stereo image super-resolution：a survey[J].Laser & Optoelectronics Progress，2021，58（18）：306-320.
[2] WANG L G，GUO Y L，LI J C，et al.NTIRE 2024 challenge on stereo image super-resolution：methods and results[C]//Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Piscataway：IEEE，2024：6198-6207.
[3] JEON D S，BAEK S H，CHOI I，et al.Enhancing the spatial resolution of stereo images using a parallax prior[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2018：1721-1730.
[4] WANG L G，WANG Y Q，LIANG Z F，et al.Learning parallax attention for stereo image super-resolution[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2020：12242-12251.
[5] SONG W，CHOI S，JEONG S，et al.Stereoscopic image super-resolution with stereo consistent feature[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence.Palo Alto：AAAI，2020：12031-12038.
[6] WANG Y Q，YING X Y，WANG L G，et al.Symmetric parallax attention for stereo image super-resolution[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Piscataway：IEEE，2021：766-775.
[7] CHEN C Q，QING C M，XU X M，et al.Cross parallax attention network for stereo image super?resolution[J].IEEE Transactions on Multimedia，2022，24：202-216.
[8] CHU X J，CHEN L Y，YU W Q.NAFSSR：stereo image super-resolution using NAFNet[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Piscataway：IEEE，2022：1238-1247.
[9] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems 30，2017：5998-6008.
[10] LIN J X，YIN L Y，WANG Y J.Steformer：efficient stereo image super-resolution with transformer[J].IEEE Transactions on Multimedia，2023，25：8396-8407.
[11] JIN K，WEI Z Q，YANG A，et al.SwiniPASSR：swin transformer based parallax attention network for stereo image super?resolution[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.Piscataway：IEEE，2022：919-928.
[12] LIU Z，LIN Y T，CAO Y，et al.Swin Transformer：hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Piscataway：IEEE，2022：9992-10002.
[13] ZHANG D Y，LIANG S，HE T，et al.CVIformer：cross-view interactive transformer for efficient stereoscopic image super-resolution[J].IEEE Transactions on Emerging Topics in Computational Intelligence，2025，9（2）：1107-1118.
[14] GU A，DAO T.Mamba：linear-time sequence modeling with selective state spaces[EB/OL].[2025?01?18].https：//arxiv.org/abs/2312.00752.
[15] WALEFFE R，BYEON W，RIACH D，et al.An empirical study of mamba-based language models[EB/OL].[2025-01-18].https：//arxiv.org/abs/2406.07887.
[16] CAO A D，FENG Z L，HU J C，et al.Vision mamba mender[C]//Advances in Neural Information Processing Systems 37，2024：51905-51929.
[17] GU A，GOEL K，Ré C.Efficiently modeling long sequences with structured state spaces[C]//Proceedings of the 10th International Conference on Learning Representations，2022.
[18] ZHU L H，LIAO B C，ZHANG Q，et al.Vision mamba：efficient visual representation learning with bidirectional state space model[C]//Proceedings of the 41st International Conference on Machine Learning，2024：62429-62442.
[19] WANG J，ZHU W T，WANG P C，et al.Selective structured state-spaces for long-form video understanding[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2023：6387-6397.
[20] LIU Y，TIAN Y J，ZHAO Y Z，et al.VMamba：visual state space model[C]//Advances in Neural Information Processing Systems 37，2024：103031-103063.
[21] GUO H，LI J M，DAI T，et al.MambaIR：a simple baseline for image restoration with state-space model[C]//Proceedings of the 18th European Conference on Computer Vision.Cham：Springer，2025：222-241.
[22] KONG L S，DONG J X，GE J J，et al.Efficient frequency domain-based transformers for high-quality image deblurring[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2023：5886-5895.
[23] SCHARSTEIN D，HIRSCHMüLLER H，KITAJIMA Y，et al.High-resolution stereo datasets with subpixel-accurate ground truth[C]//Proceedings of the 36th German Conference on Pattern Recognition.Cham：Springer，2014：31-42.
[24] WANG Y Q，WANG L G，YANG J G，et al.Flickr1024：a large-scale dataset for stereo image super-resolution[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop.Piscataway：IEEE，2019：3852-3857.
[25] GEIGER A，LENZ P，URTASUN R.Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2012：3354-3361.
[26] MENZE M，GEIGER A.Object scene flow for autonomous vehicles[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition.Piscataway：IEEE，2015：3061-3070.
[27] LOSHCHILOV I，HUTTER F.SGDR：stochastic gradient descent with warm restarts[C]//Proceedings of the 5th International Conference on Learning Representations，2017.
[28] XU Q Y，WANG L G，WANG Y Q，et al.Deep bilateral learning for stereo image super-resolution[J].IEEE Signal Processing Letters，2021，28：613-617.
[29] DAI Q Y，LI J C，YI Q S，et al.Feedback network for mutually boosted stereo image super-resolution and disparity estimation[C]//Proceedings of the 29th ACM International Conference on Multimedia.New York：ACM，2021：1985-1993.
[30] CHEN W F，NI S J，SHAO F.CVGSR：stereo image super-resolution with cross-view guidance[J].Displays，2024，83：102736.
[31] GUO H S，LI J C，GAO G W，et al.PFT-SSR：parallax fusion transformer for stereo image super-resolution[C]//Proceedings of the 2023 IEEE International Conference on Acoustics，Speech and Signal Processing.Piscataway：IEEE，2023：1-5.