Clay Mineral Image Classification Using Fusion of Improved Residual Network and Attention Mechanism

doi:10.3778/j.issn.1002-8331.2308-0344

Abstract

Abstract: Clay minerals are hydrated aluminum silicate minerals that are widely present in sedimentary basins. Clay mineral identification using scanning electron microscopy (SEM) requires diverse and easily distinguishable characteristic information of clay minerals. However, the feature extraction capability of convolutional neural networks is still insufficient, and there is a need to enhance the expressive power of the features. Lower-level features usually contain basic information and details of images, while higher-level feature maps are more abstract and contain more abstract information. Therefore, a Res2Net50-based network with an efficient channel attention (ECA) module is proposed to make the model focus more on lower-level features and improve performance, and the last layer feature map is replaced by a multi-head self-attention (MHSA) module, which enhances feature representation ability. By combining the feature extraction capability of CNNs and the content self-attention mechanism of Transformers, the model’s feature representation capability is enhanced, surpassing that of Res2Net50. The results show that the proposed EM-Res2Net (ECA-MHSA-Res2Net50) has better performance in mineral identification of 128×128 pixel input, with a recognition accuracy of 92.85% as tested and compared with other models. This paper demonstrates that introducing ECA and MHSA modules into the model can fully extract image features and improve accuracy and efficiency.

Key words: deep learning, mineral identification, image classification, multi-scale features, channel attention, self-attention

摘要： 黏土矿物是广泛存在于沉积盆地中的含水铝硅酸盐矿物，通过扫描电子显微镜（scanning electron microscopy，SEM）获取黏土矿物在此基础上进行识别，需要获取黏土矿物的多样性和易于分辨的特征信息，然而卷积神经网络其特征提取能力仍存在不足，特征表达能力亟须增强。底层特征通常包含了图像的基本信息和细节，高层次的特征图抽象程度较高，包含了更为抽象化的信息。因此提出了一种以Res2Net50为主干网络，引入高效通道注意力（efficient channel attention，ECA）模块使模型更关注底层特征，从而提升模型性能，最后一层采用多头注意力（multi-head self-attention，MHSA）模块学习到不同层次的特征表示。采用混合方式同时利用了CNN的特征提取能力、Transformer的内容自注意力机制提升模型的特征表达能力，取得了优于Res2Net50的性能。结果表明：提出的EM-Res2Net（ECA-MHSA-Res2Net50）对128×128像素输入的矿物识别效果更为理想，经测试识别准确率为92.85%，通过与其他模型对比分析，该研究结果证明引入ECA模块和MHSA模块后的模型能够充分提取图像的特征，具有更高的准确率和时效性。

关键词: 深度学习, 矿物识别, 图像分类, 多尺度特征, 通道注意力, 自注意力

DU Ruishan, CHEN Yuxin, MENG Lingdong, ZHANG Tong, CHENG Jiaxin. Clay Mineral Image Classification Using Fusion of Improved Residual Network and Attention Mechanism[J]. Computer Engineering and Applications, 2024, 60(23): 333-339.

杜睿山, 陈雨欣, 孟令东, 张桐, 程佳薪. 融合改进残差网络和注意力的黏土矿物图像分类[J]. 计算机工程与应用, 2024, 60(23): 333-339.

References

[1] 孙庆峰, 陈发虎, COLIN C, 等. 黏土矿物在气候环境变化研究中的应用进展[J]. 矿物学报, 2011, 31(1): 146-152.
SUN Q F, CHEN F H, COLIN C, et al. Application progress of clay minerals in the researches of climate and environment[J]. Acta Mineralogica Sinica, 2011, 31(1): 146-152.
[2] 胡圆圆, 胡再元. 扫描电镜在碎屑岩储层黏土矿物研究中的应用[J]. 四川地质学报, 2012, 32(1): 25-28.
HU Y Y, HU Z Y. The application of SEM to the study of clay minerals from clastic rock reservoir[J]. Acta Geologica Sichuan, 2012, 32(1): 25-28.
[3] 葛磊, 李娟, 彭飚. 鄂尔多斯砒砂岩微观结构特征研究[J]. 西部大开发 (土地开发工程研究), 2019, 4(8): 36-42.
GE L, LI J, PENG B. Study on microstructure character-istics of soft rocks in Ordos[J]. Land Development and Engineering Research, 2019, 4(8): 36-42.
[4] 王海勇, 潘海涛, 刘贵楠. 融合注意力机制和课程式学习的人脸识别方法[J]. 计算机科学与探索, 2023, 17(8): 1893-1903.
WANG H Y, PAN H T, LIU G N. Face recognition method based on attention mechanism and curriculum learning[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(8): 1893-1903.
[5] WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2020: 11531-11539.
[6] SRINIVAS A, LIN T Y, PARMAR N, et al. Bottleneck transformers for visual recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 16519-16529.
[7] 胡嫚嫚. 基于改进的卷积神经网络和多尺度特征融合的乳腺疾病图像分类[D]. 上海: 东华大学, 2022.
HU M M. Breast disease image classification based on improved convolutional neural network and multi-scale feature fusion[D]. Shanghai: Donghua University, 2022.
[8] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[9] SZEGEDY C, LIU W, JIA Y, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 1-9.
[10] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[11] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[12] HUANG G, LIU Z, MAATEN V D L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
[13] LI C, WANG D. Application of machine learning techniques in mineral classification for scanning electron microscope-energy dispersive X-ray spectroscopy (SEM-EDS) images[J]. Journal of Petroleum Science and Engineering, 2020, 200(3): 108178.
[14] ZHANG Y, LI M, HAN S, et al. Intelligent identification for rock-mineral microscopic images using ensemble machine learning algorithms[J]. Sensors, 2019, 19(18): 3914.
[15] CHEN Z H. Deep learning-based method for SEM image segmentation in mineral characterization, an example from Duvernay shale samples in western Canada sedimentary basin[J]. Computers & Geosciences, 2020, 138: 104450.
[16] LIU C, LI M, ZHANG Y, et al. An enhanced rock mineral recognition method integrating a deep learning model and clustering algorithm[J]. Minerals, 2019, 9(9): 516.
[17] 刘珏先, 滕奇志, 王正勇, 等. 基于协同表示的多特征融合岩石分类[J]. 计算机应用, 2016, 36(3): 854-858.
LIU J X, TENG Q Z, WANG Z Y, et al. Rock classification of multi-feature fusion based on collaborative representation[J]. Journal of Computer Applications, 2016, 36(3): 854-858.
[18] 白林, 魏昕, 刘禹, 等. 基于VGG模型的岩石薄片图像识别[J]. 地质通报, 2019, 38(12): 2053-2058.
BAI L, WEI X, LIU Y, et al. Rock thin section image recognition and classification based on VGG model[J]. Geological Bulletin of China, 2019, 38(12): 2053-2058.
[19] 白林, 姚钰, 李双涛, 等. 基于深度学习特征提取的岩石图像矿物成分分析[J]. 中国矿业, 2018, 27(7): 178-182.
BAI L, YAO Y, LI S T, et al. Mineral composition analysis of rock image based on deep learning feature extraction[J]. China Mining Magazine, 2018, 27(7): 178-182.
[20] 彭伟航, 白林, 商世为, 等. 基于改进Inception V3模型的常见矿物智能识别[J]. 地质通报, 2019, 38(12): 2059-2066.
PENG W H, BAI L, SHANG S W, et al. Common mineral intelligent recognition based on improved InceptionV3[J]. Geological Bulletin of China, 2019, 38 (12): 2059-2066.
[21] 郭艳军, 周哲, 林贺洵, 等. 基于深度学习的智能矿物识别方法研究[J]. 地学前缘, 2020, 27(5): 39-47.
GUO Y J, ZHOU Z, LIN H X, et al. The mineral intelligence identification method based on deep learning algorithms[J]. Earth Science Frontiers, 2020, 27(5): 39-47.
[22] LIU Y, ZHANG Z, LIU X, et al. Ore image classification based on small deep learning model: evaluation and optimization of model depth, model structure and data size[J]. Minerals Engineering, 2021, 172: 107020.
[23] ZHOU W, WANG H, WAN Z. Ore image classification based on improved CNN[J]. Computers & Electrical Engineering, 2022, 99: 107819.
[24] GAO S H, CHENG M M, ZHAO K, et al. Res2Net: a new multi-scale backbone architecture[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021: 43(2): 652-662.
[25] 王燕, 王振宇. 改进Res2Net和注意力机制的高光谱图像分类[J]. 计算机工程与应用, 2023, 59(19): 151-158.
WANG Y, WANG Z Y. Hyperspectral image classification based on Res2Net and attention mechanism[J]. Computer Engineering and Applications, 2023, 59(19): 151-158.
[26] 董乙杉, 郭靖圆, 李明泽, 等. 基于反向瓶颈和LCBAM设计的X光违禁品检测[J]. 计算机科学与探索, 2024, 18(5): 1259-1270.
DONG Y S, GUO J Y, LI M Z, et al. An X-ray prohibited items detection model based on inverted bottleneck and attention mechanism[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(5): 1259-1270.
[27] MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention[C]//Advances in Neural Information Processing Systems, 2014: 2204-2212.
[28] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017: 5998-6008.
[29] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision, 2018: 3-19.
[30] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141.
[31] 祁宣豪, 智敏. 图像处理中注意力机制综述[J]. 计算机科学与探索, 2024, 18(2): 345-362.
QI X H, ZHI M. A review of attention mechanisms in image processing[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(2): 345-362.
[32] 黄英来, 姜忠良. 改进残差网络甜瓜叶片病害的识别研究[J]. 计算机工程与应用, 2024, 60(15): 189-197.
HUANG Y L, JIANG Z L. Research on identification of melon leaf diseases with improved residual network[J]. Computer Engineering and Applications, 2024, 60(15): 189-197.