SFA-ConvNeXt：Dermoscopic Image Classification for Stepwise Aggregation of Multiscale ConvNeXt

doi:10.3778/j.issn.1002-8331.2306-0386

Abstract

Abstract: Early detection of skin cancer significantly improves the five-year survival rate for patients. However, due to the subtle nature of early malignant tumors in the skin, their symptoms are not apparent, and specialized doctors need to perform multiple biopsies and extract lesion tissues to diagnose the type of lesion. Traditional artificial intelligence methods have low accuracy in identifying skin lesion images primarily because they are difficult to simultaneously focus on spatial details and shallow semantic features. To effectively represent spatial positions and shallow feature information, and avoid the model being overly concerned with detailed information, which can lead to misclassification of easily distinguishable images, a progressive aggregation attention network based on ConvNeXt is proposed. This method utilizes a hierarchical ConvNeXt encoder to extract deep and shallow features of lesion regions layer by layer. By employing parallel spatial attention, it effectively integrates spatial position information with deep or shallow semantic features, aggregating multi-scale contextual information. Meanwhile, a progressive feature aggregation module is designed to effectively integrate deep and shallow features and aggregate them by dynamically adjusting weights, closely aligning with the process of rough observation and meticulous examination in skin image classification by specialized doctors. Experimental tests on the ISIC2018 and ISIC2019 datasets show that the accuracy, precision, recall, and F1-Score of this method are 95.27%, 93.76%, 92.83%, and 93.18%, respectively, for ISIC2018, and 92.63%, 91.06%, 87.05%, and 88.81%, respectively, for ISIC2019. Compared to ConvNeXt, the accuracy　is improved by 2.13 and 3.29 percentage points respectively, demonstrating its ability to effectively extract detailed and rough features, providing new evidence for the diagnosis of skin image through dermatoscopy.

Key words: ConvNeXt, parallel spatial attention, stepwise feature aggregation, dermatoscopic images, image classification

摘要： 皮肤癌的早期发现对患者的五年生存率有着显著的提高，然而由于早期恶性肿瘤在皮肤中的病变非常细微，其症状并不明显，专业医生需要进行多次活检并提取病变组织才可以诊断出病变类型。现有的机器学习方法由于难以同时关注空间细节信息与浅层语义特征，其在皮肤病变图像中识别准确率并不高。为了有效表示空间位置和浅层特征信息，避免模型过于关注细节信息导致易分图像误分类等问题，提出了一种基于ConvNeXt的逐级聚合注意力网络。该方法通过分层ConvNeXt编码器逐层提取病变区域的深层和浅层特征，并通过并行空间注意力有效整合空间位置信息和深层或浅层语义特征，聚合多尺度上下文信息。同时，设计逐级特征聚合模块有效整合深层和浅层特征，并通过动态调整权重的方式将深层和浅层特征聚合，高度符合专业医生对皮肤镜图像分类时粗略观察和细微观察的过程。在ISIC2018、ISIC2019数据集中上进行实验测试，其准确率、精确率、召回率和F1-Score分别是95.27%、93.76%、92.83%、93.18%与92.63%、91.06%、87.05%、88.81%。通过与ConvNeXt相比，准确率分别提升了2.13和3.29个百分点，证明其能有效地提取细节特征和粗略特征，为皮肤镜图像的诊断提供新的依据。

关键词: ConvNeXt, 并行空间注意力, 逐级特征聚合, 皮肤镜图像, 图像分类

WANG Zetong, ZHANG Junhua, WANG Xiao. SFA-ConvNeXt：Dermoscopic Image Classification for Stepwise Aggregation of Multiscale ConvNeXt[J]. Computer Engineering and Applications, 2024, 60(20): 244-253.

王泽彤, 张俊华, 王肖. SFA-ConvNeXt：逐级聚合多尺度ConvNeXt的皮肤镜图像分类[J]. 计算机工程与应用, 2024, 60(20): 244-253.

References

[1] KUMARI S, CHOUDHARY P K, SHUKLA R, et al. Recent advances in nanotechnology based combination drug therapy for skin cancer[J]. Journal of Biomaterials Science, Polymer Edition, 2022, 33(11): 1435-1468.
[2] ANAND V, GUPTA S, NAYAK S R, et al. An automated deep learning models for classification of skin disease using dermoscopy images: a comprehensive study[J]. Multimedia Tools and Applications, 2022, 81(26): 37379-37401.
[3] CAZZATO G, MANGIALARDI K, FALCICCHIO G, et al. Preferentially expressed antigen in melanoma (PRAME) and human malignant melanoma: a retrospective study[J]. Genes, 2022, 13(3): 545.
[4] GUY JR G P, THOMAS C C, THOMPSON T, et al. Vital signs: melanoma incidence and mortality trends and projections—United States, 1982—2030[J]. MMWR Morbidity and Mortality Weekly Report, 2015, 64(21): 591-596.
[5] SONTHALIA S, PASQUALI P, AGRAWAL M, et al. Dermoscopy update: review of its extradiagnostic and expanding indications and future prospects[J]. Dermatology Practical & Conceptual, 2019, 9(4): 253-264.
[6] MOHAKUD R, DASH R. Skin cancer image segmentation utilizing a novel EN-GWO based hyper-parameter optimized FCEDN[J]. Journal of King Saud University-Computer and Information Sciences, 2022, 34(10): 9889-9904.
[7] LODDO A, PUTZU L. On the reliability of CNNs in clinical practice: a computer-aided diagnosis system case study[J]. Applied Sciences, 2022, 12(7): 3269.
[8] ESTEVA A, KUPREL B, NOVOA R A, et al. Dermatologist-level classification of skin cancer with deep neural networks[J]. Nature, 2017, 542(7639): 115-118.
[9] KHOULOUD S, AHLEM M, FADEL T, et al. W-net and inception residual network for skin lesion segmentation and classification[J]. Applied Intelligence, 2022: 1-19.
[10] LOPEZ A R, GIRO-I-NIETO X, BURDICK J, et al. Skin lesion classification from dermoscopic images using deep learning techniques[C]//Proceedings of the 2017 13th IASTED International Conference on Biomedical Engineering, 2017: 49-54.
[11] ALANAZI S A. Melanoma identification through X-ray modality using inception-v3 based convolutional neural network[J]. Computers, Materials & Continua, 2022, 72(1): 37-55.
[12] XIA X, XU C, NAN B. Inception-v3 for flower classification[C]//Proceedings of the 2017 2nd International Conference on Image, Vision and Computing, 2017: 783-787.
[13] NIDA N, IRTAZA A, YOUSAF M H. A novel region-extreme convolutional neural network for melanoma malignancy recognition[J]. Mathematical Problems in Engineering, 2021: 1-18.
[14] ALMARAZ-DAMIAN J A, PONOMARYOV V, SADOVNYCHIY S, et al. Melanoma and nevus skin lesion classification using handcraft and deep learning feature fusion via mutual information measures[J]. Entropy, 2020, 22(4): 484.
[15] ZHANG J, XIE Y, XIA Y, SHEN C. Attention residual learning for skin lesion classification[J]. IEEE Transactions on Medical Imaging, 2019, 38: 2092-2103.
[16] AFZA F, SHARIF M, KHAN M A, et al. Multiclass skin lesion classification using hybrid deep features selection and extreme learning machine[J]. Sensors, 2022, 22(3): 799.
[17] LIU Z, MAO H, WU C Y, et al. A convNet for the 2020s[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 11976-11986.
[18] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[19] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision, 2018: 3-19.
[20] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2980-2988.
[21] HUANG G, LIU Z, MAATEN V D L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
[22] LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 9992-10002.
[23] RYALI C, HU Y T, BOLYA D, et al. Hiera: a hierarchical vision transformer without the bells-and-whistles[J]. arXiv:2306.00989, 2023.
[24] ALI R, HARDIE R C, NARAYANAN B N, et al. Deep learning ensemble methods for skin lesion analysis towards melanoma detection[C]//Proceedings of the 2019 IEEE National Aerospace and Electronics Conference, 2019: 311-316.
[25] PACHECO A G C, ALI A R, TRAPPENBERG T. Skin cancer detection based on deep learning and entropy to detect outlier samples[J]. arXiv:1909.04525, 2019.
[26] ALI R, HARDIE R C, DE SILVA M S, et al. Skin lesion segmentation and classification for ISIC 2018 by combining deep CNN and handcrafted features[J]. arXiv:1908.05730, 2019.
[27] AHMED S A A, YANIKO?LU B, G?KSU ?, et al. Skin lesion classification with deep CNN ensembles[C]//Proceedings of the 2020 28th Signal Processing and Communications Applications Conference, 2020: 1-4.
[28] GUISSOUS A E. Skin lesion classification using deep neural network[J]. arXiv:1911.07817, 2019.
[29] BENYAHIA S, MEFTAH B, LéZORAY O. Multi-features extraction based on deep learning for skin lesion classification[J]. Tissue and Cell, 2022, 74: 101701.
[30] 张慧婷. 用于糖尿病视网膜病变分级的深度学习模型研究[D]. 太原: 太原理工大学, 2021.
ZHANG H T. The research of deep learning model for diabetic retinopathy grading[D]. Taiyuan: Taiyuan University of Technology, 2021.
[31] SUN R, LI Y, ZHANG T, et al. Lesion-aware transformers for diabetic retinopathy grading[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 10938-10947.
[32] KASSANI S H, KASSANI P H, KHAZAEINEZHAD R, et al. Diabetic retinopathy classification using a modified xception architecture[C]//Proceedings of the 2019 IEEE International Symposium on Signal Processing and Information Technology, 2019: 1-6.