Few-Shot Scene Classification with Attention Mechanism in Remote Sensing

doi:10.3778/j.issn.1002-8331.2301-0012

Abstract

Abstract: Remote sensing scene classification is a hot research topic in the field of computer vision, and it is of great significance to semantic understanding of remote sensing images. At present, remote sensing scene classification methods based on deep learning occupy a dominant position in this field. However, it suffers from the lack of samples and poor model generalization ability in actual application scenarios. Therefore, this paper proposes a few-shot remote scene classification method based on attention mechanism, and designs a structure of dual-branches similarity measurement. This method is based on the meta-learning training strategy to divide the dataset into tasks. At the meantime, the input images are divided into blocks in order to preserve the feature distribution in the remote sensing image. Then the lightweight attention module is introduced into the feature extraction network to reduce the risk of overfitting and ensure the acquisition of discriminative features. Finally, based on earth mover’s distance (EMD), a dual-branches similarity measurement module is added to improve the discriminative ability of the classifier. The results show that compared with the classic small-sample learning method, the few-shot remote scene classification method proposed in this paper can significantly improve the classification performance.

Key words: remote sensing scene classification, few-shot learning, meta-learning, attention mechanism, dual-branches similarity measurement

摘要： 遥感图像场景分类是计算机视觉领域的热点研究方向，对遥感图像场景及其语义理解意义重大。目前，基于深度学习的遥感图像场景分类方法在该领域占据主导地位。然而实际应用场景面临着样本数据较少、模型泛化能力较差的问题，致使基于深度学习的遥感图像场景分类方法实现难度较大，性能大幅下降。针对上述难点，提出了基于注意力机制的小样本遥感图像场景分类方法，设计了一种双分支判别结构进行相似性度量。该方法基于元学习训练策略对数据集进行任务制划分；为最大限度保留遥感图像中的特征分布，对输入图像进行重叠分块；在特征提取网络中引入轻量级注意力模块，降低过拟合风险并保证判别性特征的获取；在EMD（earth mover’s distance）距离的基础上设计添加双分支相似性度量模块，提升分类器的判别能力。实验结果表明，相较于经典小样本学习方法，所提出的小样本遥感图像场景分类方法能够显著提升分类性能。

关键词: 遥感图像场景分类, 小样本学习, 元学习, 注意力机制, 双分支判别

ZHANG Duona, ZHAO Hongjia, LU Yuanyao, CUI Jian, ZHANG Baochang. Few-Shot Scene Classification with Attention Mechanism in Remote Sensing[J]. Computer Engineering and Applications, 2024, 60(4): 173-182.

张多纳, 赵宏佳, 鲁远耀, 崔健, 张宝昌. 融入注意力机制的小样本遥感图像场景分类[J]. 计算机工程与应用, 2024, 60(4): 173-182.

References

[1] TAO C, LU W, QI J, et al. Spatial information considered network for scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(6): 984-988.
[2] ANWER R M, KHAN F S, VAN DE WEIJER J, et al. Binary patterns encoded convolutional neural networks for texture recognition and remote sensing scene classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 138: 74-85.
[3] 蓝金辉, 王迪, 申小盼. 卷积神经网络在视觉图像检测的研究进展[J]. 仪器仪表学报, 2020, 41(4): 167-182.
LAN J H, WANG D, SHEN X P. Research progress on visual image detection based on convolutional neural network[J]. Chinese Journal of Scientific Instrument, 2020, 41(4): 167-182.
[4] 李炳臻, 刘克, 顾佼佼,等. 卷积神经网络研究综述[J]. 计算机时代, 2021(4): 8-12.
LI B Z, LIU K, GU J J, et al. Review of the researches on convolutional neural networks[J]. Computer Era, 2021(4): 8-12.
[5] 刘腊梅, 王晓娜, 刘万军, 等. 融合转置卷积与深度残差图像语义分割方法[J]. 计算机科学与探索, 2022, 16(9): 2132-2142.
LIU L M, WANG X N, LIU W J, et al. Image semantic segmentation method with fusion of transposed convolution and deep residual[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2132-2142.
[6] 欧阳柳, 贺禧, 瞿绍军. 全卷积注意力机制神经网络的图像语义分割[J]. 计算机科学与探索, 2022, 16(5): 1136-1145.
OUYANG L, HE X, QU S J. Fully convolutional neural network with attention module for semantic segmentation[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1136-1145.
[7] 张哲晗, 方薇, 杜丽丽, 等. 基于编码-解码卷积神经网络的遥感图像语义分割[J]. 光学学报, 2020, 40(3): 40-49.
ZHANG Z H, FANG W, DU L L, et al. Semantic segmentation of remote sensing image based on encoder-decoder convolutional neural network[J]. Acta Optica Sinica, 2020, 40(3): 40-49.
[8] CHENG G, YANG C, YAO X, et al. When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(5): 2811-2821.
[9] PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[10] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005.
[11] LEMKE C, BUDKA M, GABRYS B. Meta-learning: a survey of trends and technologies[J]. Artificial Intelligence Review, 2015, 44(1): 117-130.
[12] YANG R, HUANG H M, HONG Q H, et al. Synaptic suppression triplet-STDP learning rule realized in second-order memristors[J]. Advanced Functional Materials, 2018, 28(5):1704455.
[13] LEE J, YANG C. Deep neural network and meta-learning-based reactive sputtering with small data sample counts[J]. Journal of Manufacturing Systems, 2022, 62: 703-717.
[14] HU J, LU J, TAN Y P. Discriminative deep metric learning for face verification in the wild[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, 2014: 1875-1882.
[15] SOHN K. Improved deep metric learning with multi-class N-pair loss objective[C]//Advances in Neural Information Processing Systems 29, 2016: 1849-1857.
[16] HUBBALLI N, KHANDAIT P. KeyClass: efficient keyword matching for network traffic classification[J]. Computer Communications, 2022, 185: 79-91.
[17] SUNG F, YANG Y, ZHANG L, et al. Learning to compare: relation network for few-shot learning[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[18] LI Z, HUANG H, ZHANG Z, et al. Manifold learning-based semisupervised neural network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5508712.
[19] ZHANG C, CAI Y, LIN G, et al. DeepEMD: few-shot image classification with differentiable earth mover’s distance and structured classifiers[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 12200-12210.
[20] YANG Y, NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the Sigspatial International Conference on Advances in Geographic Information Systems, 2010: 270.
[21] XIA G S, HU J, HU F, et al. AID: a benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3965-3981.
[22] AFRASIYABI A, LALONDE J F, GAGNé C. Mixture-based feature space learning for few-shot image classification[C]//Proceedings of the 2021 International Conference on Computer Vision, 2021.
[23] WU S, ZHONG S, LIU Y. Deep residual learning for image steganalysis[J]. Multimedia Tools and Applications, 2018, 77: 10437-10453.
[24] HU J, WANG H, WANG J, et al. SA-Net: a scale-attention network for medical image segmentation[J]. PLoS One, 2021, 16(4): e0247388.
[25] RUBNER Y, TOMASI C, GUIBAS L J. The earth mover’s distance as a metric for image retrieval[J]. International Journal of Computer Vision, 2000, 40(2): 99-121.
[26] CAI L, LI H, DONG W, et al. Micro-expression recognition using 3D DenseNet fused squeeze-and-excitation networks[J]. Applied Soft Computing, 2022, 119: 108594.
[27] FINN C, ABBEEL P, LEVINE S. Model-agnostic meta-learning for fast adaptation of deep networks[C]//Proceedings of the 34th International Conference on Machine Learning, 2017: 1126-1135.