Super Resolution Reconstruction of Hyperspectral Remote Sensing Image Based on Shunt

doi:10.3778/j.issn.1002-8331.2201-0173

Abstract

Abstract: Hyperspectral image（HSI） contains a large number of spectral bands per pixel, while the spatial resolution of HSI is low. The spatial resolution is effectively improved by hyperspectral image super-resolution technology. In order to solve the problem that the spatial information of hyperspectral remote sensing image is unevenly distributed and occupies the same amount of computation during super-resolution reconstruction, which leads to the lack of refinement of reconstruction work, a multi-level shunt and detail enhancement framework for hyperspectral remote sensing image super-resolution reconstruction is proposed in this study. That is, the hyperspectral remote sensing image is pre shunted through the sub image shunting network. The maximum number of large squares that allow exchange with details to display high and low frequency details and use networks from different sources to change locations. The reconstruction work is more detailed and specific, and the reconstruction effect and performance are improved. Experiments show that this method is superior to the traditional CNN based method in visual quality, index measurement and classification application, and the SRE and PSNR indexes are improved by 4.18% and 9.35% respectively.

Key words: hyperspectral, remote sensing images, super resolution, convolutional neural network

摘要： 高光谱图像（hyperspectral image，HSI）每个像素包含大量光谱带，而HSI的空间分辨率较低。高光谱图像超分辨率技术可以有效提高空间分辨率。为了解决高光谱遥感图像的空间信息分布不均匀，超分辨率重建时占据相同计算量导致重建工作不够细化的问题，提出了一种多层级分流和细节增强的高光谱遥感图像超分辨率重建框架。即通过子图分流网络对高光谱遥感图像进行预先分流，改进带细节增强的多尺度Retinex算法对图像的高低频信息进行分离，再使用不同复杂程度的分支网络分别进行重建，重建工作更加细化具体，提高重建效果和性能。实验证明该方法可以在视觉质量、指标测量和分类应用方面优于传统的基于CNN的方法，在SRE和MPSNR指标方面分别提高了4.18%和9.35%。

关键词: 高光谱, 遥感图像, 超分辨率, 卷积神经网络

XU Yinghao, LYU Yuchao, LIU Sifan, ZHU Xijun. Super Resolution Reconstruction of Hyperspectral Remote Sensing Image Based on Shunt[J]. Computer Engineering and Applications, 2022, 58(18): 260-267.

徐英豪, 吕玉超, 刘斯凡, 朱习军. 基于分流的高光谱遥感图像超分辨重建[J]. 计算机工程与应用, 2022, 58(18): 260-267.

References

[1] AKGUN T，ALTUNBASAK Y，MERSEREAU R M.Super-resolution reconstruction of hyperspectral images[J].IEEE Transactions on Image Processing，2005，14（11）：1860-1875.
[2] GOU S，LIU S，YANG S，et al.Remote sensing image super-resolution reconstruction based on nonlocal pairwise dictionaries and double regularization[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing，2014，7（12）：4784-4792.
[3] GLASNER D，BAGON S，IRANI M.Super-resolution from a single image[C]//2009 IEEE 12th International Conference on Computer Vision，2009：349-356.
[4] DONG C，LOY C C，HE K，et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，38（2）：295-307.
[5] 席志红，侯彩燕，袁昆鹏，等.基于深层残差网络的加速图像超分辨率重建[J].光学学报，2019，39（2）：89-98.
XI Z H，HOU C Y，YUAN K P，et al.Super-resolution reconstruction of accelerated image based on deep residual network[J].Acta Optica Sinica，2019，39（2）：89-98.
[6] 刘超，张晓晖，胡清平.图像超分辨率卷积神经网络加速算法[J].国防科技大学学报，2019，41（2）：91-97.
LIU C，ZHANG X H，HU Q P.Image super resolution convolution neural network acceleration algorithm[J].Journal of National University of Defense Technology，2019，41（2）：91-97.
[7] JIANG D H，ZHANG S Z，DAI L，et al.Multi-scale generative adversarial network for image super-resolution[J].Soft Computing，2022，26：3631-3641.
[8] LEDIG C，THEIS L，HUSZáR F，et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：4681-4690.
[9] 彭晏飞，高艺，杜婷婷，等.生成对抗网络的单图像超分辨率重建方法[J].计算机科学与探索，2020，14（9）：1612-1620.
PENG Y F，GAO Y，DU T T，et al.Single image super-resolution reconstruction method for generative adversarial network[J].Journal of Frontiers of Computer Science and Technology，2020，14（9）：1612-1620.
[10] 蒋明峰，支明豪，李杨，等.基于自注意力机制生成对抗网络的超分辨率磁共振图像重建[J].中国科学（信息科学），2021，51（6）：959-970.
JIANG M F，ZHI M H，LI Y，et al.Super-resolution re-construction of MR image with self-attention based generate adversarial network algorithm[J].Scientia Sinica（Informationis），2021，51（6）：959-970.
[11] ROMANO Y，ISIDORO J，MILANFAR P.RAISR：rapid and accurate image super resolution[J].IEEE Transactions on Computational Imaging，2016，3（1）：110-125.
[12] KONG X，ZHAO H，QIAO Y，et al.ClassSR：a general framework to accelerate super-resolution networks by data characteristic[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：12016-12025.
[13] AHN N，KANG B，SOHN K A.Fast，accurate，and lightweight super-resolution with cascading residual network[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：252-268.
[14] 段立娟，武春丽，恩擎，等.基于小波域的深度残差网络图像超分辨率算法[J].软件学报，2019，30（4）：941-953.
DUAN L J，WU C L，EN Q，et al.Deep residual network in wavelet domain for image super-resolution[J].Journal of Software，2019，30（4）：941-953.
[15] 邹沛煜，张卫东，史金余，等.基于高低频分量融合的水下图像增强算法[J].激光与光电子学进展，2020，57（16）：131-142.
ZOU P Y，ZHANG W D，SHI J Y，et al.Underwater image enhancement algorithm based on fusion of high and low frequency components[J].Laser & Optoelectronics Progress，2020，57（16）：131-142.
[16] 李欣，韦宏卫，张洪群.结合深度学习的单幅遥感图像超分辨率重建[J].中国图象图形学报，2018，23（2）：209-218.
LI X，WEI H W，ZHANG H Q.Super-resolution recon-struction of single remote sensing image combined with deep learning[J].Journal of Image and Graphics，2018，23（2）：209-218.
[17] WANG X，MA J，JIANG J.Hyperspectral image super-resolution via recurrent feedback embedding and spatial-spectral consistency regularization[J].IEEE Transactions on Geoscience and Remote Sensing，2021，60：1-13.
[18] 周圆，王明非，杜晓婷，等.基于层次特征复用的视频超分辨率重建[J/OL].自动化学报：1-14[2022-01-07].DOI：10.
16383/j.aas.c210095.
ZHOU Y，WANG M F，DU X T，et al.Video super-resolution via hierarchical feature reuse[J/OL].Acta Automatica Sinica：1-14[2022-01-07].DOI：10.16383/j.aas.c210095.
[19] 李金新，黄志勇，李文斌，等.基于多层次特征融合的图像超分辨率重建[J/OL].自动化学报：1-11[2022-01-07].DOI：10.16383/j.aas.c200585.
LI J X，HUANG Z Y，LI W B，et al.Image super-resolution based on multi hierarchical features fusion network[J/OL].Acta Automatica Sinica：1-11[2022-01-07].DOI：10.16383/j.aas.c200585.
[20] 杨宏业.卷积神经网络的多光谱遥感图像超分辨率重建[D].徐州：中国矿业大学，2020.
YANG H Y.Multi-spectral remote sensing images super-resolution based on convolutional neural network[D].Xuzhou：China University of Mining and Technology，2020.
[21] 张毅锋，刘袁，蒋程，等.用于超分辨率重建的深度网络递进学习方法[J].自动化学报，2020，46（2）：274-282.
ZHANG Y F，LIU Y，JIANG C，et al.A curriculum learning approach for single image super resolution[J].Acta Automatica Sinica，2020，46（2）：274-282.
[22] LAI W S，HUANG J B，AHUJA N，et al.Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the IEEE Conference on Comuter Vision and Pattern Recognition，2017：624-632.