Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (18): 111-121.DOI: 10.3778/j.issn.1002-8331.2102-0092

• Generative Adversarial Networks • Previous Articles     Next Articles

Generation and Classification of Skin Cancer Images Based on Self-Attention StyleGAN

ZHAO Chen, SHUAI Renjun, MA Li, LIU Wenjia, WU Menglin   

  1. 1.College of Computer Science and Technology, Nanjing Tech University, Nanjing 211816, China
    2.Nanjing Health Information Center, Nanjing 210003, China
    3.Changzhou No.2 People’s Hospital Affiliated to Nanjing Medical University, Changzhou, Jiangsu 213003, China
  • Online:2022-09-15 Published:2022-09-15

基于Self-Attention StyleGAN的皮肤癌图像生成与分类


  1. 1.南京工业大学 计算机科学与技术学院,南京 211816
    2.南京市卫生信息中心,南京 210003
    3.南京医科大学附属常州第二人民医院,江苏 常州 213003

Abstract: Aiming at the problem of skin cancer classification tasks represented by melanoma, there is an imbalance in the number and weight of various samples in the data set, and the poor quality of skin cancer samples generated by the existing confrontation generation network makes it difficult to distinguish in clinical diagnosis. A skin cancer image sample generation and classification framework based on self-attention-based style generation confrontation network(Self-Attention-StyleGAN) combined with SE-ResNeXt-50. This framework introduces a self-attention mechanism on the basis of StyleGAN, redesigns the generator’s style control and noise input structure, and reconstructs the discriminator to adjust the image generator. Effectively synthesize high-quality skin damage images. Skin cancer sample images are classified by using SE-ResNeXt-50 to better extract the information of different hierarchical feature maps of sample images, thereby improving the balanced multi-class accuracy(BMA). The experimental results show that the sample image quality generated by this model on the ISIC2019 skin cancer dataset is high, and the classification BMA reaches 94.71%. This method improves the accuracy of skin lesion image classification, helps dermatologists to judge and diagnose different types of skin lesions, and analyzes different stages and difficult to distinguish skin lesions.

Key words: melanoma, skin lesion image generation and classification, ResNeXt, StyleGAN, deep convolutional neural networks

摘要: 针对以黑色素瘤为代表的皮肤癌分类任务存在数据集各类样本数量、权重不均衡,且现有的对抗生成网络生成的皮肤癌样本图像质量较差导致临床诊断时难以分辨等问题,提出了一种基于自注意力的样式生成对抗网络(Self-Attention StyleGAN)与SE-ResNeXt-50相结合的皮肤癌图像样本生成与分类框架。该框架在样式生成对抗网络(StyleGAN)的基础上引入了自注意力机制,对生成器的样式控制和噪声输入结构进行了重新设计,并重构了鉴别器对图像生成器进行了调整,从而有效地合成高质量的皮肤癌病变图像。使用SE-ResNeXt-50来对皮肤癌样本图像进行分类,更好地提取样本图像不同层次特征图的信息,从而提高了平衡多类精度(BMA)。实验结果表明,该模型在ISIC2019皮肤癌数据集上生成的样本图像质量较高,且分类BMA达到94.71%。该方法提高了皮肤癌病变图像分类的准确性,帮助皮肤科医生对不同类型的皮肤癌病变进行判断和诊断,并对不同阶段和难以区分的皮肤癌病变进行分析。

关键词: 黑色素瘤, 皮肤癌病变图像生成与分类, ResNeXt, StyleGAN, 深度卷积神经网络