Generation and Classification of Skin Cancer Images Based on Self-Attention StyleGAN

doi:10.3778/j.issn.1002-8331.2102-0092

Abstract

Abstract: Aiming at the problem of skin cancer classification tasks represented by melanoma, there is an imbalance in the number and weight of various samples in the data set, and the poor quality of skin cancer samples generated by the existing confrontation generation network makes it difficult to distinguish in clinical diagnosis. A skin cancer image sample generation and classification framework based on self-attention-based style generation confrontation network（Self-Attention-StyleGAN） combined with SE-ResNeXt-50. This framework introduces a self-attention mechanism on the basis of StyleGAN, redesigns the generator’s style control and noise input structure, and reconstructs the discriminator to adjust the image generator. Effectively synthesize high-quality skin damage images. Skin cancer sample images are classified by using SE-ResNeXt-50 to better extract the information of different hierarchical feature maps of sample images, thereby improving the balanced multi-class accuracy（BMA）. The experimental results show that the sample image quality generated by this model on the ISIC2019 skin cancer dataset is high, and the classification BMA reaches 94.71%. This method improves the accuracy of skin lesion image classification, helps dermatologists to judge and diagnose different types of skin lesions, and analyzes different stages and difficult to distinguish skin lesions.

Key words: melanoma, skin lesion image generation and classification, ResNeXt, StyleGAN, deep convolutional neural networks

摘要： 针对以黑色素瘤为代表的皮肤癌分类任务存在数据集各类样本数量、权重不均衡，且现有的对抗生成网络生成的皮肤癌样本图像质量较差导致临床诊断时难以分辨等问题，提出了一种基于自注意力的样式生成对抗网络（Self-Attention StyleGAN）与SE-ResNeXt-50相结合的皮肤癌图像样本生成与分类框架。该框架在样式生成对抗网络（StyleGAN）的基础上引入了自注意力机制，对生成器的样式控制和噪声输入结构进行了重新设计，并重构了鉴别器对图像生成器进行了调整，从而有效地合成高质量的皮肤癌病变图像。使用SE-ResNeXt-50来对皮肤癌样本图像进行分类，更好地提取样本图像不同层次特征图的信息，从而提高了平衡多类精度（BMA）。实验结果表明，该模型在ISIC2019皮肤癌数据集上生成的样本图像质量较高，且分类BMA达到94.71%。该方法提高了皮肤癌病变图像分类的准确性，帮助皮肤科医生对不同类型的皮肤癌病变进行判断和诊断，并对不同阶段和难以区分的皮肤癌病变进行分析。

关键词: 黑色素瘤, 皮肤癌病变图像生成与分类, ResNeXt, StyleGAN, 深度卷积神经网络

ZHAO Chen, SHUAI Renjun, MA Li, LIU Wenjia, WU Menglin. Generation and Classification of Skin Cancer Images Based on Self-Attention StyleGAN[J]. Computer Engineering and Applications, 2022, 58(18): 111-121.

赵宸, 帅仁俊, 马力, 刘文佳, 吴梦麟. 基于Self-Attention StyleGAN的皮肤癌图像生成与分类[J]. 计算机工程与应用, 2022, 58(18): 111-121.

References

[1] FEIGELSON H S，POWERS J D，KUMAR M，et al.Melanoma incidence，recurrence，and mortality in an integrated healthcare system：a retrospective cohort study[J].Cancer Medicine，2019，8（9）：4508-4516.
[2] FONG Z V，TANABE K K.Comparison of melanoma guidelines in the USA，Canada，Europe，Australia and New Zealand：a critical appraisal and comprehensive review[J].British Journal of Dermatology，2014，170（1）：20-30.
[3] CONFORTI C，GIUFFRIDA R，VEZZONI R，et al.Dermoscopy and the experienced clinicians[J].International Journal of Dermatology，2020，59（1）：16-22.
[4] ZALAUDEK I，ARGENZIANO G，DI STEFANI A，et al.Dermoscopy in general dermatology[J].Dermatology，2006，212（1）：7-18.
[5] BRAUN R P，RABINOVITZ H S，OLIVIERO M，et al.Dermoscopy of pigmented skin lesions[J].Journal of the American Academy of Dermatology，2005，52（1）：109-121.
[6] AL-MASNI M A，AL-ANTARI M A，PARK H M，et al.A deep learning model integrating FrCN and residual convolutional networks for skin lesion segmentation and classification[C]//Proceedings of the 2019 IEEE Eurasia Conference on Biomedical Engineering，Healthcare and Sustainability（ECBIOS），2019.
[7] 吴晓光，邓文强，牛小辰，等.基于条件生成对抗网络的人体步态生成[J].仪器仪表学报，2020，41（1）：132-140.
WU X G，DENG W Q，NIU X C，et al.Gait generation of human based on the conditional generative adversarial networks[J].Chinese Journal of Scientific Instrument，2020，41（1）：132-140.
[8] 武随烁，杨金福，单义，等.使用孪生注意力机制的生成对抗网络的研究[J].计算机科学与探索，2020，14（5）：833-840.
WU S S，YANG J F，SHAN Y，et al.Research on generative adversarial networks using twins attention mechanism[J].Journal of Frontiers of Computer Science and Technology，2020，14（5）：833-840.
[9] MILTON M A A.Automated skin lesion classification using ensemble of deep neural networks in isic 2018：skin lesion analysis towards melanoma detection challenge[J].arXiv：1901.10802，2019.
[10] AKRAM T，LODHI H M J，NAQVI S R，et al.A multilevel features selection framework for skin lesion classification[J].Human-Centric Computing and Information Sciences，2020，10：1-26.
[11] HAIGEN H，XIANGYONG K，QIANWEI Z，et al.Melanoma classification method by integrating deep convolutional residual network[J].Computer Science，2019，46（5）：247-253.
[12] RASHID H，TANVEER M A，KHAN H A.Skin lesion classification using GAN based data augmentation[C]//Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society（EMBC），2019.
[13] BAUR C，ALBARQOUNI S，NAVAB N.MelanoGANs：high resolution skin lesion synthesis with GANs[J].arXiv：1804.04338，2018.
[14] BAUR C，ALBARQOUNI S，NAVAB N.Generating highly realistic images of skin lesions with GANs[J].arXiv：1809.01410，2018.
[15] BISSOTO A，PEREZ F，VALLE E，et al.Skin lesion synthesis with generative adversarial networks[J].arXiv：1902.
03253，2019.
[16] BISSOTO A，VALLE E，AVILA S.The six fronts of the generative adversarial networks[J].arXiv：1910.13076，2019.
[17] GONG A，YAO X，LIN W.Dermoscopy image classification based on styleGANs and decision fusion[J].IEEE Access，2020，8：70640-70650.
[18] KARRAS T，LAINE S，AILA T.A style-based generator architecture for generative adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019.
[19] TORREY L，SHAVLIK J.Transfer learning[M]//Handbook of research on machine learning applications and trends：algorithms，methods，and techniques.[S.l.]：IGI Global，2010：242-264.
[20] PAN S J，YANG Q.A survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering，2009，22（10）：1345-1359.
[21] RUSSAKOVSKY O，DENG J，SU H，et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision，2015，115（3）：211-252.
[22] HU J，SHEN L，ALBANIE S，et al.Squeeze-and-excitation networks[J].arXiv：1709.01507，2017.
[23] XIE S，GIRSHICK R，DOLLáR P，et al.Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017.
[24] LIU W，WEN Y，YU Z，et al.Sphereface：deep hypersphere embedding for face recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017.
[25] COMBALIA M，CODELLA N C，ROTEMBERG V，et al.BCN20000：dermoscopic lesions in the wild[J].arXiv：1908.02288，2019.
[26] CODELLA N C，GUTMAN D，CELEBI M E，et al.Skin lesion analysis toward melanoma detection：a challenge at the 2017 international symposium on biomedical imaging（isbi），hosted by the international skin imaging collaboration（isic）[C]//Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging（ISBI 2018），2018.
[27] TSCHANDL P，ROSENDAHL C，KITTLER H.The HAM10000
dataset，a large collection of multi-source dermatoscopic images of common pigmented skin lesions[J].Scientific Data，2018，5：180161.
[28] SALIMANS T，GOODFELLOW I，ZAREMBA W，et al.Improved techniques for training gans[C]//Proceedings of the Advances in Neural Information Processing Systems，2016.
[29] RUSSAKOVSKY O，DENG J，SU H，et al.Imagenet large scale visual recognition challenge[J].International of Journal of Computer Vision，2015，115（3）：211-252.
[30] HEUSEL M，RAMSAUER H，UNTERTHINER T，et al.Gans trained by a two time-scale update rule converge to a local nash equilibrium[C]//Proceedings of the Advances in Neural Information Processing Systems，2017.
[31] KARRAS T，AILA T，LAINE S，et al.Progressive growing of gans for improved quality，stability，and variation[J].arXiv：1710.10196，2017.
[32] RADFORD A，METZ L，CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].arXiv：1511.06434，2015.
[33] DENTON E L，CHINTALA S，FERGUS R.Deep generative image models using a laplacian pyramid of adversarial networks[C]//Proceedings of the Advances in Neural Information Processing Systems，2015.
[34] GOODFELLOW I，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Proceedings of the Advances in Neural Information Processing Systems，2014.
[35] GUISSOUS A E.Skin lesion classification using deep neural network[J].arXiv：1911.07817，2019.
[36] PACHECO A G，ALI A R，TRAPPENBERG T.Skin cancer detection based on deep learning and entropy to detect outlier samples[J].arXiv：1909.04525，2019.
[37] NAHATA H，SINGH S P.Deep learning solutions for skin cancer detection and diagnosis[M]//Machine learning with health care perspective.[S.l.]：Springer，2020：159-182.
[38] GESSERT N，NIELSEN M，SHAIKH M，et al.Skin lesion classification using ensembles of multi-resolution efficientNets with meta data[J].MethodsX，2020：100864.