Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (10): 22-34.DOI: 10.3778/j.issn.1002-8331.2211-0082
• Research Hotspots and Reviews • Previous Articles Next Articles
JIANG Zhongmin, ZHANG Wanyan, WANG Wenju
Online:
2023-05-15
Published:
2023-05-15
姜中敏,张婉言,王文举
JIANG Zhongmin, ZHANG Wanyan, WANG Wenju. Research of Deep Learning-Based Computational Spectral Imaging for Single RGB Image[J]. Computer Engineering and Applications, 2023, 59(10): 22-34.
姜中敏, 张婉言, 王文举. 单幅RGB图像计算光谱成像的深度学习研究综述[J]. 计算机工程与应用, 2023, 59(10): 22-34.
Add to citation manager EndNote|Ris|BibTeX
URL: http://cea.ceaj.org/EN/10.3778/j.issn.1002-8331.2211-0082
[1] HU Z,FANG C,LI B,et al.First-in-human liver-tumour surgery guided by multispectral fluorescence imaging in the visible and near-infrared-i/ii windows[J].Nature Biomedical Engineering,2020,4(3):259-271. [2] SEGARRA J,BUCHAILLOT M L,ARAUS J L,et al.Remote sensing for precision agriculture:sentinel-2 improved features and applications:5[J].Agronomy,2020,10(5):641. [3] SHI N,LI G,LEI E,et al.Hyperspectral imaging to Chinese paintings at the palace museum[J].Sciences of Conservation and Archaeology,2017,29(3):23-29. [4] SHIMONI M,HAELTERMAN R,PERNEEL C.Hypersectral imaging for military and security applications:combining myriad processing and sensing techniques[J].IEEE Geoscience and Remote Sensing Magazine,2019,7(2):101-117. [5] GREEN R O,EASTWOOD M L,SARTURE C M,et al.Imaging spectroscopy and the airborne visible/infrared imaging spectrometer(aviris)[J].Remote Sensing of Environment,1998,65(3):227-248. [6] JAMES J.Spectrograph design fundamentals[M].Cambridge:Cambridge University Press,2007. [7] 张春光.基于超光谱成像系统的声光可调滤波技术研究[D].哈尔滨:哈尔滨工业大学,2008. ZHANG C H.The technology of the acousto-optical tunable filter based on the hyperspectral imaging system[D].Harbin:Harbin Institute of Technology,2008. [8] CAO X,DU H,TONG X,et al.A prism-mask system for multispectral video acquisition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(12):2423-2435. [9] WAGADARIKAR A,JOHN R,WILLETT R,et al.Single disperser design for coded aperture snapshot spectral imaging[J].Applied Optics,2008,47(10):44-51. [10] DONOHO D L.Compressed sensing[J].IEEE Transactions on Information Theory,2006,52(4):1289-1306. [11] ABED F M,AMIRSHAHI S H,ABED M R M.Reconstruction of reflectance data using an interpolation technique[J].Journal of the Optical Society of America-Optics Image Science and Vision,2009,26(3):613-624. [12] ZHU Y,LI B,XU X.Spectral reconstruction and accuracy appraisal based on pseudo inverse method[C]//2012 Symposium on Photonics and Optoelectronics,2012:1-3. [13] ELRIFAI I,MAHGOUB H,MAGDY M,et al.Enhanced spectral reflectance reconstruction using pseudo-inverse estimation method[J].International Journal of Image Processing,2013,7(3):278-285. [14] ARAD B,BEN-SHAHAR O.Sparse recovery of hyperspectral signal from natural RGB images[C]//14th European Conference on Computer Vision.Cham:Springer,2016:19-34. [15] WANG L,XIONG Z,SHI G,et al.Adaptive nonlocal sparse representation for dual-camera compressive hyperspectral imaging[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(10):2104-2111. [16] ZHANG S,WANG L,FU Y,et al.Computational hyperspectral imaging based on dimension-discriminative low-rank tensor recovery[C]//2019 IEEE/CVF International Conference on Computer Vision,2019:10182-10191. [17] ARAD B,BEN-SHAHAR O,TIMOFTE R,et al.NTIRE 2018 challenge on spectral reconstruction from RGB images[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2018:1042-104209. [18] ARAD B,TIMOFTE R,BEN-SHAHAR O,et al.NTIRE 2020 challenge on spectral reconstruction from an RGB image[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:1806-1822. [19] ARAD B,TIMOFTE R,YAHEL R,et al.NTIRE 2022 spectral recovery challenge and data set[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2022:862-880. [20] RONNEBERGER O,FISCHER P,BROX T.U-Net:convolutional networks for biomedical image segmentation[C]//18th International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham:Springer,2015:234-241. [21] SHI W,CABALLERO J,HUSZáR F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition,2016:1874-1883. [22] MEI Y,FAN Y,ZHOU Y,et al.Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:5689-5698. [23] ZHANG J,SU R,FU Q,et al.A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging[J].Scientific Reports,2022,12:11905. [24] YASUMA F,MITSUNAGA T,ISO D,et al.Generalized assorted pixel camera:postcapture control of resolution,dynamic range,and spectrum[J].IEEE Transactions on Image Processing,2010,19(9):2241-2253. [25] CHAKRABARTI A,ZICKLER T.Statistics of real-world hyperspectral images[C]//2011 IEEE Conference on Computer Vision and Pattern Recognition,2011:193-200. [26] NGUYEN R M H,PRASAD D K,BROWN M S.Training-based spectral reconstruction from a single RGB image[C]//13th European Conference on Computer Vision.Cham:Springer,2014:186-201. [27] CHOI I,JEON D S,NAM G,et al.High-quality hyperspectral reconstruction using a spectral prior[J].ACM Transactions on Graphics,2017,36(6):218. [28] SHRESTHA R,PILLAY R,GEORGE S,et al.Quality evaluation of spectral imaging:quality factors and metrics[J].Journal of the International Colour Association,2014,12:22-35. [29] YUHAS R H,GOETZ A F H,BOARDMAN J W.Discrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm[C]//Summaries of the Third Annual JPL Airborne Geoscience Workshop,1992:147-149. [30] ROMERO J,GARCíA-BELTRáN A,HERNáNDEZ-ANDRéS J.Linear bases for representation of natural and artificial illuminants[J].Journal of The Optical Society of America A,1997,14(5):1007-1014. [31] GALLIANI S,LANARAS C,MARMANIS D,et al.Learned spectral super-resolution[J].arXiv:1703.09470,2017. [32] KOUNDINYA S,SHARMA H,SHARMA M,et al.2D-3D CNN based architectures for spectral reconstruction from RGB images[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2018:957-964. [33] ZHANG L,LANG Z,WANG P,et al.Pixel-aware deep function-mixture network for spectral super-resolution[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence,2020:12821-12828. [34] STIEBEL T,KOPPERS S,SELTSAM P,et al.Reconstructing spectral images from RGB-images using a convolutional neural network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2018:948-953. [35] FUBARA B J,SEDKY M,DYKE D.RGB to spectral reconstruction via learned basis functions and weights[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:1984-1993. [36] ZHU Z,LIU H,HOU J,et al.Deep amended gradient descent for efficient spectral reconstruction from single RGB images[J].IEEE Transactions on Computational Imaging,2021,7:1176-1188. [37] YAN L,WANG X,ZHAO M,et al.Reconstruction of hyperspectral data from RGB images with prior category information[J].IEEE Transactions on Computational Imaging,2020,6:1070-1081. [38] CAN Y B,TIMOFTE R.An efficient CNN for spectral reconstruction from RGB images[J].arXiv:1804.04647,2018. [39] XIONG Z,SHI Z,LI H,et al.HSCNN:CNN-based hyperspectral image recovery from spectrally undersampled projections[C]//2017 IEEE International Conference on Computer Vision Workshops,2017:518-525. [40] SHI Z,CHEN C,XIONG Z W,et al.HSCNN+:advanced CNN-based hyperspectral recovery from RGB images[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2018:1052-1058. [41] NG P C,VERDIE Y,LU J,et al.Efficient hyperspectral reconstruction from RGB images with line-pixel deconvolution[C]//2022 IEEE 14th Image,Video,and Multidimensional Signal Processing Workshops,2022:1-5. [42] LI J,DU S,WU C,et al.DRCR net:dense residual channel re-calibration network with non-local purification for spectral super resolution[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2022:1258-1267. [43] ALVAREZ GILA A,V AN DE WEIJER J,GARROTE E,et al.Adversarial networks for spatial context-aware spectral image reconstruction from RGB[C]//2017 16th IEEE International Conference on Computer Vision,2017:480-490. [44] MIAO X,Y UAN X,PU Y,et al.Lambda-Net:reconstruct hyperspectral images from a snapshot measurement[C]//2019 IEEE/CVF International Conference on Computer Vision,2019:4058-4068. [45] LIU X,GHERBI A,WEI Z,et al.Multispectral image reconstruction from color images using enhanced variational autoencoder and generative adversarial network[J].IEEE Access,2021,9:1666-1679. [46] LIU L,LEI S,SHI Z,et al.Hyperspectral remote sensing imagery generation from RGB images based on joint discrimination[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2021,14:7624-7636. [47] ISOLA P,ZHU J Y,ZHOU T,et al.Image-to-image translation with conditional adversarial networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition,2017:5967-5976. [48] WANG X,GIRSHICK R,GUPTA A,et al.Non-local neural networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2018:7794-7803. [49] LIU P,ZHAO H.Adversarial networks for scale feature-attention spectral image reconstruction from a single RGB[J].Sensors,2020,20(8):E2426. [50] BANERJEE A,PALRECHA A.MXR-U-nets for real time hyperspectral reconstruction[J].arXiv:2004.07003,2020. [51] 宋蓓蓓,马穗娜,何帆,等.Res2-Unet深度学习网络的RGB-高光谱图像重建[J].光学精密工程,2022,30(13):1606-1619. SONG P P,MA H N,HE F,et al.Hyperspectral reconstruction from RGB images based on Res2-Unet deep learning network[J].Optical Precision Engineering,2022,30(13):1606-1619. [52] ZHAO Y,PO L-M,YAN Q,et al.Hierarchical regression network for spectral reconstruction from RGB images[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:1695-1704. [53] PENG H,CHEN X,ZHAO J.Residual pixel attention network for spectral reconstruction from RGB images[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:2012-2020. [54] KAYA B,CAN Y B,TIMOFTE R.Towards spectral estimation from a single RGB image in the wild[C]//2019 IEEE/CVF International Conference on Computer Vision Workshops,2019:3546-3555. [55] LI J,WU C,SONG R,et al.Adaptive weighted attention network with camera spectral sensitivity prior for spectral reconstruction from RGB images[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2020:1894-1903. [56] GAO X,WANG T,YANG J,et al.Deep-learning-based hyperspectral imaging through a RGB camera[J].Journal of Electronic Imaging,2021,30:053014. [57] KOHEI Y,HAN X-H.Deep residual attention network for hyperspectral image reconstruction[C]//2020 25th International Conference on Pattern Recognition,2021:8547-8553. [58] NATHAN D S,UMA K,VINOTHINI D S,et al.Light weight residual dense attention net for spectral reconstruction from RGB images[J].arXiv:2004.06930,2020. [59] WANG W,WANG J.Double ghost convolution attention mechanism network:a framework for hyperspectral reconstruction of a single RGB image:2[J].Sensors,2021,21(2):666. [60] HAN K,WANG Y,TIAN Q,et al.GhostNet:more features from cheap operations[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:1577-1586. [61] FU Y,ZHANG T,WANG L,et al.Coded hyperspectral image reconstruction using deep external and internal learning[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,44(7):3404-3420. [62] ZHANG T,FU Y,WANG L,et al.Hyperspectral image reconstruction using deep external and internal learning[C]//2019 IEEE/CVF International Conference on Computer Vision,2019:8559-8568. [63] VASWANI A,SHAZEER N,PARMAR N,et a.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems,2017:6000-6010. [64] CHEN C F R,FAN Q,PANDA R.CrossViT:cross-attention multi-scale vision transformer for image classification[C]//2021 IEEE/CVF International Conference on Computer Vision,2021:347-356. [65] DAI Z,CAI B,LIN Y,et al.UP-DETR:unsupervised pre-training for object detection with transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:1601-1610. [66] STRUDEL R,GARCIA R,LAPTEV I,et al.Segmenter:transformer for semantic segmentation[C]//2021 IEEE/CVF International Conference on Computer Vision,2021:7242-7252. [67] LI K,WANG S,ZHANG X,et al.Pose recognition with cascade transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:1944-1953. [68] YANG S,QUAN Z,NIE M,et al.TransPose:keypoint localization via transformer[C]//2021 IEEE/CVF International Conference on Computer Vision,2021:11782-11792. [69] CAI Y,LIN J,HU X,et al.Mask-guided spectral-wise transformer for efficient hyperspectral image reconstruction[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022:17481-17490. [70] CAI Y,LIN J,HU X,et al.Coarse-to-fine sparse transformer for hyperspectral image reconstruction[C]//17th European Conference on Computer Vision.Cham:Springer,2022:686-704. [71] CAI Y,LIN J,LIN Z,et al.MST++:multi-stage spectral-wise transformer for efficient spectral reconstruction[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops,2022:744-754. [72] 金秋雨.基于卷积神经网络的多光谱图像重建技术研究[D].沈阳:沈阳工业大学,2021. JIN Q Y.Research on multispectral image reconstruction technology based on convolutional neural network[D].Shenyang:Shenyang University of Technology,2021. [73] MA L,RATHGEB A,MUBARAK H,et al.Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging[J].Journal of Biomedical Optics,2022,27(5):059801. [74] SUN Y,YANG Y,LIU Q,et al.Unsupervised spatial-spectral network learning for hyperspectral compressive snapshot reconstruction[J].IEEE Transactions on Geoscience and Remote Sensing,2022,60:5514314. [75] LIU Z,ZHENG Y,HAN X H.Deep unsupervised fusion learning for hyperspectral image super resolution[J].Sensors,2021,21(7):2348. [76] GANIN Y,USTINOVA E,AJAKAN H,et al.Domain-adversarial training of neural networks[J].The Journal of Machine Learning Research,2016,17(l):59. [77] 施展.基于深度学习的高光谱图像空谱超分辨率重建[D].合肥:中国科学技术大学,2020. SHI Z.Deep learning based spatial and spectral super-resolution reconstruction of hyperspectral images[D].Hefei:University of Science and Technology of China,2020. [78] MEHTA A,SINHA H,MANDAL M,et al.Domain-aware unsupervised hyperspectral reconstruction for aerial image dehazing[C]//2021 IEEE Winter Conference on Applications of Computer Vision,2021:413-422. [79] ZHU Z,LIU H,HOU J,et al.Semantic-embedded unsupervised spectral reconstruction from single RGB images in the wild[C]//2021 IEEE/CVF International Conference on Computer Vision,2021:2259-2268. [80] LI D,LI S,ZHU M,et al.Unsupervised data fidelity en-hancement network for spectral CT reconstruction[C]//SPIE 11312,Medical Imaging 2020:Physics of Medical Imaging,2020:1081-1088. [81] MARTíNEZ E,CASTRO S,BACCA J,et al.Efficient transfer learning for spectral image reconstruction from RGB images[C]//2020 IEEE Colombian Conference on Applications of Computational Intelligence,2020:1-6. [82] GREFF K,BELLETTI F,BEYER L,et al.Kubric:a scalable dataset generator[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022:3739-3751. [83] MARTINEZ J,SHEWAKRAMANI J,WEI LIU T,et al.Permute,quantize,and fine-tune:efficient compression of neural networks[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:15694-15703. [84] FAN L,DING Y,FAN D,et al.GrainSpace:a large-scale dataset for fine-grained and domain-adaptive recognition of cereal grains[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022:21084-21093. [85] LIU Y,ZHANG W,XIANG C,et al.Learning to affiliate:mutual centralized learning for few-shot classification[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022:14391-14400. [86] CHEN S,HONG Z,XIE G S,et al.MSDN:mutually semantic distillation network for zero-shot learning[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022:7602-7611. [87] FRANKLE J,CARBIN M.The lottery ticket hypothesis:finding sparse,trainable neural networks[J].arXiv:1803. 03635,2018. [88] LOU Q,GUO F,KIM M,et al.AutoQ:automated kernel-wise neural network quantization[J].arXiv:1902.05690,2019. [89] GOU J,YU B,MAYBANK S J,et al.Knowledge distillation:a survey[J].International Journal of Computer Vision,2021,129(6):1789-1819. [90] KOREN Y,BELL R,VOLINSKY C.Matrix factorization techniques for recommender systems[J].Computer,2009,42(8):30-37. [91] WANG Q,LI D,HUANG X,et al.Optimizing FFT-based convolution on ARMv8 multi-core CPUs[C]//26th International Conference on Parallel and Distributed Computing.Cham:Springer,2020:248-262. [92] MAZZIA V,SALVETTI F,CHIABERGE M.Efficient-CapsNet:capsule network with self-attention routing:1[J].Scientific Reports,2021,11(1):14634. [93] WANG X,LIAN L,YU S X.Unsupervised visual attention and invariance for reinforcement learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:6673-6683. [94] CAO H,TAN C,GAO Z,et al.A survey on generative diffusion model[J].arXiv:2209.02646,2022. |
[1] | CHEN Jishang, Abudukelimu Halidanmu, LIANG Yunze, Abulizi Abudukelimu, Aishan Mikelayi, GUO Wenqiang. Review of Application of Deep Learning in Symbolic Music Generation [J]. Computer Engineering and Applications, 2023, 59(9): 27-45. |
[2] | JIANG Qiuxiang, GUO Weipeng, WANG Zilong, OUYANG Xingtao, LONG Ruirui. Application and Prospect of Python Language in Field of Hydrology and Water Resources [J]. Computer Engineering and Applications, 2023, 59(9): 46-58. |
[3] | LUO Huilan, CHEN Han. Spatial-Temporal Convolutional Attention Network for Action Recognition [J]. Computer Engineering and Applications, 2023, 59(9): 150-158. |
[4] | DAI Chao, LIU Ping, SHI Juncai, REN Hongjie. Regularized Extraction of Remotely Sensed Image Buildings Using U-Shaped Networks [J]. Computer Engineering and Applications, 2023, 59(8): 105-116. |
[5] | LIU Hualing, PI Changpeng, ZHAO Chenyu, QIAO Liang. Review of Cross-Domain Object Detection Algorithms Based on Depth Domain Adaptation [J]. Computer Engineering and Applications, 2023, 59(8): 1-12. |
[6] | HE Jiafeng, CHEN Hongwei, LUO Dehan. Review of Real-Time Semantic Segmentation Algorithms for Deep Learning [J]. Computer Engineering and Applications, 2023, 59(8): 13-27. |
[7] | ZHANG Yanqing, MA Jianhong, HAN Ying, CAO Yangjie, LI Jie, YANG Cong. Review of Research on Real-World Single Image Super-Resolution Reconstruction [J]. Computer Engineering and Applications, 2023, 59(8): 28-40. |
[8] | WEI Jian, ZHAO Xu, LI Lianpeng. Siamese Network Weak Target Tracking Algorithm Fused with Location Information Attention [J]. Computer Engineering and Applications, 2023, 59(7): 198-206. |
[9] | ZHAO Hongwei, ZHENG Jiajun, ZHAO Xinxin, WANG Shengchun, LI Yidong. Rail Surface Defect Method Based on Bimodal-Modal Deep Learning [J]. Computer Engineering and Applications, 2023, 59(7): 285-293. |
[10] | WANG Jing, JIN Yuchu, GUO Ping, HU Shaoyi. Survey of Camera Pose Estimation Methods Based on Deep Learning [J]. Computer Engineering and Applications, 2023, 59(7): 1-14. |
[11] | JIANG Yuying, CHEN Xinyu, LI Guangming, WANG Fei, GE Hongyi. Graph Neural Network and Its Research Progress in Field of Image Processing [J]. Computer Engineering and Applications, 2023, 59(7): 15-30. |
[12] | ZHOU Yurong, ZHANG Qiaoling, YU Guangzeng, XU Weiqiang. Review of Acoustic Signal-Based Industrial Equipment Fault Diagnosis [J]. Computer Engineering and Applications, 2023, 59(7): 51-63. |
[13] | WANG Xiaoming, MAO Yushi, XU Bin, WANG Zilei. Content Structure Preserved Image Style Transfer Method [J]. Computer Engineering and Applications, 2023, 59(6): 146-154. |
[14] | LYU Xiaoling, YANG Shengyue, ZHANG Minglu, LIANG Ming, WANG Junchao. Improved Fisheye Image Target Detection Algorithm Based on YOLOv5 Network [J]. Computer Engineering and Applications, 2023, 59(6): 241-250. |
[15] | PENG Pei, ZHANG Meiling, ZHENG Dong. Side Channel Attack Fused with CNN_LSTM [J]. Computer Engineering and Applications, 2023, 59(6): 268-276. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||