Remote Sensing Image Super-Resolution Reconstruction Method for Ship Target Recognition

doi:10.3778/j.issn.1002-8331.2304-0200

Abstract

Abstract: The degradation of space-based remote sensing image resolution poses great challenges for the recognition of ship targets. Image super-resolution reconstruction technology can provide rich information for recognition tasks. However, if image super-resolution reconstruction and ship target recognition tasks are performed independently, the internal coherence between the two tasks will be ignored. Aiming at these problems, in order to explore the effective combination of image super-resolution reconstruction and target recognition tasks, a remote sensing image super-resolution reconstruction method for ship target recognition is proposed. Specifically, a full channel concatenation network is firstly designed, which replaces the residual connection with an adaptively weighted full channel concatenation, improves the fluidity and expression performance of each layer feature, and realizes efficient super-resolution reconstruction of remote sensing images. In order to further explore the potential of super-resolution reconstruction to improve the performance of ship target recognition, a joint network of super-resolution reconstruction and target recognition is proposed by introducing multi-task learning technology. The stable training of the joint end-to-end network is realized by multi-stage training optimization strategy, so as to guide effective mutual supervised learning between tasks. The experimental results on the public data set FGSCR-42 show that when the resolution of remote sensing images is degraded by 8 times and 16 times, the proposed super-resolution reconstruction network helps the accuracy of ship target recognition to increase by 33.27 and 17.48 percentage points respectively; the proposed joint network further improves the recognition accuracy by 1.75 and 1.91 percentage points.

Key words: space-based remote sensing images, image super-resolution reconstruction, ship target recognition

摘要： 天基遥感图像分辨率的退化为舰船目标的识别带来了巨大挑战。图像超分辨率重建技术可以为识别任务提供丰富的信息，然而将图像超分辨率重建与舰船目标识别任务分别独立进行，将会忽略两个任务间的内在相关性。针对这些问题，为了探索图像超分辨率重建与目标识别任务间的有效结合方式，提出了面向舰船目标识别的遥感图像超分辨率重建方法。设计了一种通道全连接网络，以自适应加权的通道全连接代替残差连接，提升各层特征的流动性与表达性能，实现遥感图像的高效超分辨率重建。为了进一步挖掘超分辨率重建对舰船目标识别性能提升的潜力，引入多任务学习技术，提出了一种超分辨率重建与目标识别联合网络，通过多阶段训练优化策略实现联合端到端网络的稳定训练，从而引导任务间进行有效的互相监督学习。在公开数据集FGSCR-42上的实验结果表明，当遥感图像在8倍和16倍的分辨率退化情况下，提出的超分辨率重建网络帮助舰船目标识别准确率分别提升了33.27和17.48个百分点，所提联合网络则将识别准确率进一步提升了1.75和1.91个百分点。

关键词: 天基遥感图像, 图像超分辨重建, 舰船目标识别

ZHANG Tianlin, PANG Zheng, CHEN Hongzhen, CHEN Shi, BIAN Chunjiang. Remote Sensing Image Super-Resolution Reconstruction Method for Ship Target Recognition[J]. Computer Engineering and Applications, 2024, 60(13): 190-199.

张天霖, 逄征, 陈红珍, 陈实, 卞春江. 面向舰船目标识别的遥感图像超分辨率重建[J]. 计算机工程与应用, 2024, 60(13): 190-199.

References

[1] 刘勇. 多源卫星舰船目标观测信息融合技术研究[D]. 长沙: 国防科技大学, 2018.
LIU Y. Research on information fusion technology of multi-source satellite observation for ships[D]. Changsha: National University of Defense Technology, 2018.
[2] 安彧. 海战场舰船目标检测与识别研究[D]. 哈尔滨: 哈尔滨工程大学, 2015.
AN Y. Research on ship target detection and recognition in sea battlefield[D]. Harbin: Harbin Engineering University, 2015.
[3] 何友, 姚力波. 天基海洋目标信息感知与融合技术研究[J]. 武汉大学学报(信息科学版), 2017, 42(11): 1530-1536.
HE Y, YAO L B. Space-based sea target information awareness and fusion[J]. Geomatics and Information Science of Wuhan University, 2017, 42(11): 1530-1536.
[4] 张帅勇, 刘美琴, 姚超, 等. 分级特征反馈融合的深度图像超分辨率重建[J]. 自动化学报, 2022, 48(4): 992-1003.
ZHANG S Y, LIU M Q, YAO C, et al. Hierarchical feature feedback network for depth super-resolution reconstruction[J]. Acta Automatica Sinica, 2022, 48(4): 992-1003.
[5] 杨才东, 李承阳, 李忠博, 等. 深度学习的图像超分辨率重建技术综述[J]. 计算机科学与探索, 2022, 16(9): 1990-2010.
YANG C D, LI C Y, LI Z B, et al. Review of image super-resolution reconstruction algorithms based on deep learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1990-2010.
[6] 钟梦圆, 姜麟. 超分辨率图像重建算法综述[J]. 计算机科学与探索, 2022, 16(5): 972-990.
ZHONG M Y, JIANG L. Review of super-resolution image reconstruction algorithms[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 972-990.
[7] NOOR D F, LI Y, LI Z, et al. Multi-scale gradient image super-resolution for preserving SIFT key points in low-resolution images[J]. Signal Processing: Image Communication, 2019, 78: 236-245.
[8] SHERMEYER J, VAN ETTEN A. The effects of super-resolution on object detection performance in satellite imagery[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.
[9] RUDER S. An overview of multi-task learning in deep neural networks[J]. arXiv:1706.05098, 2017.
[10] GONG T, LEE T, STEPHENSON C, et al. A comparison of loss weighting strategies for multi task learning in deep neural networks[J]. IEEE Access, 2019, 7: 141627-141632.
[11] VANDENHENDE S, GEORGOULIS S, VAN GANSBEKE W, et al. Multi-task learning for dense prediction tasks: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(7): 3614-3633.
[12] WANG A, SINGH A, MICHAEL J, et al. GLUE: a multi-task benchmark and analysis platform for natural language understanding[J]. arXiv:1804.07461, 2018.
[13] DENG L, HINTON G, KINGSBURY B. New types of deep neural network learning for speech recognition and related applications: an overview[C]//Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, 2013: 8599-8603.
[14] CUI Z, QI G J, GU L, et al. Multitask aet with orthogonal tangent regularity for dark object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 2553-2562.
[15] DAI D, WANG Y, CHEN Y, et al. Is image super-resolution helpful for other vision tasks?[C]//Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 2016: 1-9.
[16] JI H, GAO Z, MEI T, et al. Vehicle detection in remote sensing images leveraging on simultaneous super-resolution[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 17(4): 676-680.
[17] RABBI J, RAY N, SCHUBERT M, et al. Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network[J]. Remote Sensing, 2020, 12(9): 1432.
[18] HE S, ZOU H, WANG Y, et al. ShipSRDet: an end-to-end remote sensing ship detector using super-resolved feature representation[C]//Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium, 2021: 3541-3544.
[19] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[20] SHI W, CABALLERO J, HUSZáR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2016: 1874-1883.
[21] DI Y, JIANG Z, ZHANG H. A public dataset for fine-grained ship classification in optical remote sensing images[J]. Remote Sensing, 2021, 13(4): 747.
[22] YANG J, FU K, WU Y, et al. Mutual-feed learning for super-resolution and object detection in degraded aerial imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-16.
[23] HE S, ZOU H, LI R, et al. Teacher-student network for low-quality remote sensing ship detection[C]//Proceedings of the 2021 IEEE International Conference on Computer Science, Artificial Intelligence and Electronic Engineering (CSAIEE), 2021: 283-287.
[24] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017: 136-144.
[25] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409.
1556, 2014.
[26] CHEN Y, BAI Y, ZHANG W, et al. Destruction and construction learning for fine-grained image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 5157-5166.