Skin Lesion Segmentation Based on Classification Activation Mapping and Visual Field Attention

doi:10.3778/j.issn.1002-8331.2206-0088

Abstract

Abstract: In dermatoscope image segmentation, the segmentation accuracy is affected by multiple factors, including image contrast, lesion size and foreign body interference. In order to improve the segmentation accuracy and solve the inaccurate segmentation of lesion boundaries, an improved DeepLab V3+ network is proposed. This improved network on the one hand generates the classification activation mapping of the original image, which is incorporated into the encoder of the network as a priori information to provide the network with accurate localization information and eliminate some interfering factors; on the other hand, it incorporates the visual field attention mechanism in the atrous spatial pyramid pooling module to realize local cross-field interaction; meanwhile, dice loss and sorting loss are combined into the loss function of the network, which makes the network pay more attention to the error of hard pixels and optimize the segmentation model. The proposed model is evaluated on ISIC-2017 and PH2 datasets, respectively, and its Jaccard index（JA） reaches 82.6% and 89.2%, and its accuracy reaches 95.2% and 96.5%, respectively. The experimental results show that the proposed model has higher segmentation sensitivity and its comprehensive segmentation performance is improved compared with other advanced networks.

Key words: medical image processing, segmentation of skin lesions, classification activation mapping, visual attention mechanism, mixed loss function, DeepLab V3+

摘要： 在皮肤镜图像分割问题中，分割精度受多重因素影响，包括图像对比度、病变大小及异物干扰等，为提高分割精度，解决病变边界分割不准等问题，提出一种改进的DeepLab V3+网络。该改进网络一方面生成原图像的类激活映射，融入到网络的编码器中作先验信息，为网络提供准确的定位信息并消除部分干扰因素；另一方面，在空洞空间金字塔模块中融合视野注意力机制，实现局部跨视野交互；同时将Dice损失和排序损失相结合作为本网络的损失函数，使网络更关注硬像素的误差，优化分割模型。分别在ISIC-2017和PH2数据集上对所提模型评估，其Jaccard指数（JA）分别达到82.6%和89.2%，准确率分别达到95.2%和96.5%，实验结果表明所提模型分割敏感度更高，综合分割性能较其他先进网络有所提升。

关键词: 医学图像处理, 皮肤病变分割, 类激活映射, 视野注意力机制, 混合损失函数, DeepLab V3+

ZHANG Yu, LIANG Fengmei, LIU Jianxia. Skin Lesion Segmentation Based on Classification Activation Mapping and Visual Field Attention[J]. Computer Engineering and Applications, 2023, 59(21): 187-194.

张宇, 梁凤梅, 刘建霞. 融合类激活映射和视野注意力的皮肤病变分割[J]. 计算机工程与应用, 2023, 59(21): 187-194.

References

[1] FONTANILLAS P，ALIPANAHI B，FURLOTTE N A，et al.Disease risk scores for skin cancers[J].Nature Communications，2021，12（1）：1-13.
[2] 王慧，戚倩倩，李雪，等.皮肤肿瘤图像自动分类的研究进展[J].计算机工程与应用，2022，58（16）：31-48.
WANG H，QI Q Q，LI X，et al.Research progress in automatic classification of skin lesions images[J].Computer Engineering and Applications，2022，58（16）：31-48.
[3] 陈海鹏，申铉京，龙建武.采用高斯拟合的全局阈值算法阈值优化框架[J].计算机研究与发展，2016，53（4）：892-903.
CHEN H P，SHEN X J，LONG J W.Threshold optimization framework of global thresholding algorithmsusing Gaussian fitting[J].Journal of Computer Research and Development，2016，53（4）：892-903.
[4] AHN E，KIM J，BI L，et al.Saliency-based lesion segmentation via background detection in dermoscopic images[J].IEEE Journal of Biomedical and Health Informatics，2017，21（6）：1685-1693.
[5] 张明慧，卢振泰，张娟，等.基于多图谱活动轮廓模型的脑部图像分割[J].计算机学报，2016，39（7）：1490-1500.
ZHANG M H，LU Z T，ZHANG J，et al.Brain image segmentation based on multiple atlas active contour model[J].Chinese Journal of Computers，2016，39（7）：1490-1500.
[6] BI L，KIM J，AHN E，et al.Dermoscopic image segmentation via multistage fully convolutional networks[J].IEEE Transactions on Biomedical Engineering，2017，64（9）：2065-2074.
[7] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Semantic image segmentation with deep convolutional nets and fully connected crfs[J].arXiv：1412.7062，2014.
[8] CHEN L C，ZHU Y，PAPANDREOU G，et al.Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the European Conference on Computer Vision（ECCV），2018：801-818.
[9] CHEN L C，PAPANDREOU G，KOKKINOS I，et al.Deeplab：semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected crfs[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，40（4）：834-848.
[10] ABRAHAM N，KHAN N M.A novel focal tversky loss function with improved attention u-net for lesion segmentation[C]//2019 IEEE 16th International Symposium on Biomedical Imaging（ISBI 2019），2019：683-687.
[11] TONG X，WEI J，SUN B，et al.ASCU-Net：attention gate，spatial and channel attention u-net for skin lesion segmentation[J].Diagnostics，2021，11（3）：501.
[12] 王雪.基于U-Net多尺度和多维度特征融合的皮肤病变分割方法[J].吉林大学学报（理学版），2021，59（1）：123-127.
WANG X.Skin lesion segmentation method based on U-Net with multi-scale and multi-dimensional feature fusion[J].Journal of Jilin University（Science Edition），2021，59（1）：123-127.
[13] ASHRAF H，WARIS A，GHAFOOR M F，et al.Melanoma segmentation using deep learning with test-time augmentations and conditional random fields[C]//Scientific Reports，2022，12（1）：1-16.
[14] ZHANG Z，SABUNCU M.Generalized cross entropy loss for training deep neural networks with noisy labels[C]//Advances in Neural Information Processing Systems，2018.
[15] MA J，CHEN J，NG M，et al.Loss odyssey in medical image segmentation[J].Medical Image Analysis，2021，71：102035.
[16] MILLETARI F，NAVAB N，AHMADI S A.V-net：fully convolutional neural networks for volumetric medical image segmentation[C]//2016 Fourth International Conference on 3D Vision（3DV），2016：565-571.
[17] XIE Y，ZHANG J，XIA Y，et al.A mutual bootstrapping model for automated skin lesion segmentation and classification[J].IEEE Transactions on Medical Imaging，2020，39（7）：2482-2493.
[18] ZHOU B，KHOSLA A，LAPEDRIZA A，et al.Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：2921-2929.
[19] CHOLLET F.Xception：deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1251-1258.
[20] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks，2020 IEEE[C]//CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020.
[21] CODELLA N C F，GUTMAN D，CELEBI M E，et al.Skin lesion analysis toward melanoma detection：a challenge at the 2017 international symposium on biomedical imaging（isbi），hosted by the international skin imaging collaboration（isic）[C]//2018 IEEE 15th International Symposium on Biomedical Imaging（ISBI 2018），2018：168-172.
[22] MENDON?A T，FERREIRA P M，MARQUES J S，et al.PH 2-A dermoscopic image database for research and benchmarking[C]//2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society（EMBC），2013：5437-5440.
[23] KRIZHEVSKY A，SUTSKEVER I，HINTON G E.Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems，2012.
[24] LIN T Y，MAIRE M，BELONGIE S，et al.Microsoft coco：common objects in context[C]//European Conference on Computer Vision.Cham：Springer，2014：740-755.
[25] GOYAL M，OAKLEY A，BANSAL P，et al.Skin lesion segmentation in dermoscopic images with ensemble deep learning methods[J].IEEE Access，2019，8：4171-4181.
[26] ZAFAR K，GILANI S O，WARIS A，et al.Skin lesion segmentation from dermoscopic images using convolutional neural network[J].Sensors，2020，20（6）：1601.