Research Progress in Automatic Classification of Skin Lesions Image

doi:10.3778/j.issn.1002-8331.2112-0125

Abstract

Abstract: Skin malignant lesions pose a great threat to patients’ health. Due to the limitations of existing diagnostic techniques such as poor accuracy and invasive operations, the clinical diagnosis of skin malignant lesions has low precision, high misdiagnosis rate, and low efficiency of diagnosis and treatment. The use of computer algorithms for automatic image classification can effectively improve the efficiency of clinical diagnosis. This paper systematically summarizes the related research work at home and abroad in recent years, and summarizes the skin image datasets and evaluation criteria commonly used for automatic skin lesion image classification model. The common models and effects of computer technology in skin lesion diagnosis are elaborated, the advantages, limitations and applicability of various methods are compared and analyzed, and the future development trend is prospected.

Key words: skin neoplasms, artificial intelligence, skin image classification, intelligent diagnosis, skin image dataset

摘要： 皮肤恶性肿瘤对患者健康有极大的威胁，由于现有诊断技术存在精准性差及有创操作等局限性，导致皮肤恶性肿瘤的临床诊断精度低，误诊率高，诊疗效率低下。使用计算机算法进行图像自动分类可以有效提高临床诊断效率。对近年来国内外相关研究工作进行了系统性归纳，总结了皮肤肿瘤图像自动分类模型常用的皮肤图像数据集和评估指标。对目前计算机技术在皮肤肿瘤诊断方面的常用模型及效果进行了详细的阐述，对比分析了各种方法的优势、局限及适用范围，并对未来发展趋势进行了展望。

关键词: 皮肤肿瘤, 人工智能, 皮肤图像分类, 智能诊断, 皮肤图像数据集

WANG Hui, QI Qianqian, LI Xue, SUN Weijia, LIU Ying, YAO Chunli. Research Progress in Automatic Classification of Skin Lesions Image[J]. Computer Engineering and Applications, 2022, 58(16): 31-48.

王慧, 戚倩倩, 李雪, 孙卫佳, 刘莹, 姚春丽. 皮肤肿瘤图像自动分类的研究进展[J]. 计算机工程与应用, 2022, 58(16): 31-48.

References

[1] 陈翔.2017年皮肤肿瘤临床进展回顾[J].皮肤病与性病，2018，40（2）：177-178.
CHENG X.A review of clinical advances in skin tumours in 2017[J].Journal of Dermatology and Venereology，2018，40（2）：177-178.
[2] KOMATSUBARA K M，JETER J，CARVAJAL R D，et al.Advances in the treatment of advanced extracutaneous melanomas and nonmelanoma skin cancers[J].American Society of Clinical Oncology Educational Book，2017，37：641-650.
[3] TING P T，KASPER R，ARLETTE J P.Metastatic basal cell carcinoma：report of two cases and literature review[J].Journal of Cutaneous Medicine and Surgery，2005，9：10-15.
[4] SAMARASINGHE V，MADAN V，LEAR J T.Management of high-risk squamous cell carcinoma of the skin[J].Expert Review of Anticancer Therapy，2011，11（5）：763-769.
[5] GARRUBBA C，DONKERS K.Skin cancer[J].Official Journal of the American Academy of Physician Assistants，2020，33（2）：49-50.
[6] MARIA J Q，CHRISTOPHER K B.Management of early-stage melanoma[J].Facial Plastic Surgery Clinics of North America，2019，27（1）：35-42.
[7] PIERRE F，BABAK A，NICHOLAS A F，et al.Disease risk scores for skin cancers[J].Nature Communications，2021，12（1）：1-13.
[8] 陈樟，王大光，骆丹.影像学检查在辅助确定皮肤肿瘤手术切缘中的应用进展[J].国际皮肤性病学杂志，2017，43（5）：277-280.
CHENG Z，WANG D G，LUO D.Application of imaging examination in auxiliary determination of surgical margins in skin tumor surgery[J].International Journal of Dermatology and Venereology，2017，43（5）：277-280.
[9] NADA E，TAIEB M，EL-FEKY M A，et al.Diagnosis of onychomycosis clinically by nail dermoscopy versus microbiological diagnosis[J].Archives of Dermatological Research，2020，312（5）：207-212.
[10] 王诗琪，刘洁.深度学习辅助皮肤影像自动分类的研究进展[J].中华皮肤科杂志，2020，53（12）：1037-1040.
WANG S Q，LIU J.Deep learning-assisted automatic classification of skin images[J].Chinese Journal of Dermatology，2020，53（12）：1037-1040.
[11] ZHU C Y，WANG Y K，CHEN H P，et al.A deep learning based framework for diagnosing multiple skin diseases in a clinical environment[J].Frontiers in Medicine，2021.
[12] 刘念，陈宏翔.人工智能在皮肤科领域的应用与发展[J].中华皮肤科杂志，2019，52（1）：63-66.
LIU N，CHEN H X.Application and development of artificial intelligence in dermatology[J].Chinese Journal of Dermatology，2019，52（1）：63-66.
[13] 陈琢，周欣，张健滔，等.人工智能领域基底细胞癌的诊治研究进展[J].第二军医大学学报，2019，40（5）：471-477.
CHEN Z，ZHOU X，ZHANG J T，et al.Advances in diagnosis and treatment of basal cell carcinoma with artificial intelligence[J].Academic Journal of Second Military Medical University，2019，40（5）：471-477.
[14] GUTMAN D，CODELLA N C F，CELEBI E，et al.Skin lesion analysis toward melanoma detection：a challenge at the international symposium on biomedical imaging（ISBI） 2016，hosted by the international skin imaging collaboration（ISIC）[J].arXiv：1605.01397，2016.
[15] CODELLA N C F，GUTMAN D，CELEBI M E，et al.Skin lesion analysis toward melanoma detection：a challenge at the 2017 international symposium on biomedical imaging（ISBI），hosted by the international skin imaging collaboration（ISIC）[C]//2018 IEEE 15th International Symposium on Biomedical Imaging，2018：168-172.
[16] CODELLA N，ROTEMBERG V，TSCHANDL P，et al.Skin lesion analysis toward melanoma detection 2018：a challenge hosted by the international skin imaging collaboration（ISIC）[J].arXiv：1902.03368，2019.
[17] TSCHANDL P，ROSENDAHL C，KITTLER H.The HAM10000 dataset，a large collection of multi-source dermatoscopic images of common pigmented skin lesions[J].Scientific Data，2018，5：180161.
[18] COMBALIA M，CODELLA N C F，ROTEMBERG V，et al.BCN20000：dermoscopic lesions in the wild[J].arXiv：1908.02288，2019.
[19] ROTEMBERG V，KURTANSKY N，BETZ-STABLEIN B，et al.A patient-centric dataset of images and metadata for identifying melanomas using clinical context[J].Scientific data，2021，8（1）：1-8.
[20] MENDONCA T，FERREIRA P M，MARQUES J S，et al.PH2—a dermoscopic image database for research and benchmarking[C]//2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society，2013：5437-5440.
[21] KAWAHARA J，DA N S，ARGENZIANO G，et al.Seven-point checklist and skin lesion classification using multitask multimodal neural nets[J].IEEE Journal of Biomedical and Health Informatics，2018，23（2）：538-546.
[22] GIOTIS I，MOLDERS N，LAND S，et al.MED-NODE：a computer-assisted melanoma diagnosis system using non-dermoscopic images[J].Expert Systems with Applications，2015，42（19）：6578-6585.
[23] BALLERINI L，FISHER R B，ALDRIDGE B，et al.A color and texture based hierarchical K-NN approach to the classification of non-melanoma skin lesions[M]//Color medical image analysis.Dordrecht：Springer，2013：63-86.
[24] HAN S S.SNU dataset+Quiz[DB/OL].（2021-08-16）[2021-11-03].https：//doi.org/10.6084/m9.figshare.6454973.v11.
[25] XIE B，HE X，ZHAO S，et al.XiangyaDerm：a clinical image dataset of asian race for skin disease aided diagnosis[M]//Large-scale annotation of biomedical data and expert label synthesis and hardware aware learning for medical imaging and computer assisted intervention.Cham：Springer，2019：22-31.
[26] VAPNIK V N，CHERVONENKIS A.A note on one class of perceptrons[J].Automation and Remote Control，1964，25（1）.
[27] BREIMAN L.Random forest[J].Machine Learning，2001，45：5-32.
[28] CHATTERJEE S，DEY D，MUNSHI S.Integration of morphological preprocessing and fractal based feature extraction with recursive feature elimination for skin lesion types classification[J].Computer Methods and Programs in Biomedicine，2019，178：201-218.
[29] TASOULIS S K，DOUKAS C N，MAGLOGIANNIS I，et al.Skin lesions characterisation utilising clustering algorithms[C]//6th Hellenic Conference on Artificial Intelligence：Theories，Models and Applications.Berlin：Springer-Verlag，2010：243-253.
[30] CHATTERJEE S，DEY D，MUNSHI S，et al.Extraction of features from cross correlation in space and frequency domains for classification of skin lesions[J].Biomedical Signal Processing and Control，2019，53：101581.
[31] DHIVYAA C R，SANGEETHA K，BALAMURUGAN M，et al.Skin lesion classification using decision trees and random forest algorithms[J].Journal of Ambient Intelligence and Humanized Computing，2020.DOI：10.1007/s12652-020-02675-8.
[32] MURUGAN A，NAIR S，PREETHI A，et al.Diagnosis of skin cancer using machine learning techniques[J].Microprocessors and Microsystems，2021，81：103727.
[33] TSCHANDL P，CODELLA N，AKAY B N，et al.Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification：an open，web-based，international，diagnostic study[J].The Lancet Oncology，2019，20（7）：938-947.
[34] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016.
[35] HEKLER A，UTIKAL J S，ENK A H，et al.Superior skin cancer classification by the combination of human and artificial intelligence[J].European Journal of Cancer，2019，120：114-121.
[36] HEKLER A，UTIKAL J S，ENK A H，et al.Pathologist-level classification of histopathological melanoma images with deep neural networks[J].European Journal of Cancer，2019，115：79-83.
[37] MARON R C，WEICHENTHAL M，UTIKAL J S，et al.Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks[J].European Journal of Cancer，2019，119：57-65.
[38] HAN S S，MOON I J，LIM W，et al.Keratinocytic skin cancer detection on the face using region-based convolutional neural network[J].JAMA dermatology，2020，156（1）：29-37.
[39] TSCHANDL P，RINNER C，APALLA Z，et al.Human computer collaboration for skin cancer recognition[J].Nature Medicine，2020，26（8）：1229-1234.
[40] HAN S S，KIM M S，LIM W，et al.Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm[J].Journal of Investigative Dermatology，2018，138（7）：1529-1538.
[41] JINNAI S，YAMAZAKI N，HIRANO Y，et al.The development of a skin cancer classification system for pigmented skin lesions using deep learning[J].Biomolecules，2020，10（8）：1123.
[42] HAN S S，PARK I，CHANG S E，et al.Augmented intelligence dermatology：deep neural networks empower medical professionals in diagnosing skin cancer and predicting treatment options for 134 skin disorders[J].Journal of Investigative Dermatology，2020，140（9）：1753-1761.
[43] FUJISAWA Y，OTOMO Y，OGATA Y，et al.Deep learning based，computer aided classifier developed with a small dataset of clinical images surpasses board certified dermatologists in skin tumour diagnosis[J].British Journal of Dermatology，2019，180（2）：373-381.
[44] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition，2015.
[45] LIU Z，XIONG R，JIANG T.Multi-level relationship capture network for automated skin lesion recognition[C]//24th International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2021：153-164.
[46] KIMESWENGER S，TSCHANDL P，NOACK P，et al.Artificial neural networks and pathologists recognize basal cell carcinomas based on different histological patterns[J].Modern Pathology，2021，34（5）：895-903.
[47] HUANG K，HE X，JIN Z，et al.Assistant diagnosis of basal cell carcinoma and seborrheic keratosis in Chinese population using convolutional neural network[J].Journal of Healthcare Engineering，2020，9：1713904.
[48] SINGHAL A，SHUKLA R，KANKAR P K，et al.Comparing the capabilities of transfer learning models to detect skin lesion in humans[J].Proceedings of the Institution of Mechanical Engineers，Part H：Journal of Engineering in Medicine，2020，234（10）：1083-1093.
[49] SZEGEDY C，VANHOUCKE V，IOFFE S，et al.Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：2818-2826.
[50] HUANG H W，HSU B W Y，LEE C H，et al.Development of a light weight deep learning model for cloud applications and remote diagnosis of skin cancers[J].The Journal of Dermatology，2021，48（3）：310-316.
[51] HUANG G，LIU Z，LAURENS V，et al.Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：2261-2269.
[52] TAN M，LE Q.Efficientnet：rethinking model scaling for convolutional neural networks[C]//36th International Conference on Machine Learning，2019：6105-6114.
[53] 谢斌，何小宇，黄伟红，等.基于卷积神经网络的基底细胞癌和色素痣的临床图像鉴别[J].中南大学学报（医学版），2019，44（9）：1063-1070.
XIE B，HE X Y，HUANG W H，et al.Clinical image identification of basal cell carcinoma and pigmented nevi based on convolutional neural network[J].Journal of Central South University（Medical Sciences），2019，44（9）：1063-1070.
[54] CHOLLET F.Xception：deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：1251-1258.
[55] MIJWIL M M.Skin cancer disease images classification using deep learning solutions[J].Multimedia Tools and Applications，2021，80（17）：26255-26271.
[56] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[J].arXiv：1409.
1556，2014.
[57] AL-MASNI M A，KIM D H，KIM T S.Multiple skin lesions diagnostics via integrated deep convolutional networks for segmentation and classification[J].Computer Methods and Programs in Biomedicine，2020，190：105351.
[58] JIANG Y Q，XIONG J H，LI H Y，et al.Recognizing basal cell carcinoma on smartphone captured digital histopathology images with a deep neural network[J].British Journal of Dermatology，2020，182（3）：754-762.
[59] KHOULOUD S，AHLEM M，FADEL T，et al.W-Net and inception residual network for skin lesion segmentation and classification[J].Applied Intelligence，2022，52（4）：3976-3994.
[60] KHAN M A，ZHANG Y D，SHARIF M，et al.Pixels to classes：intelligent learning framework for multiclass skin lesion localization and classification[J].Computers & Electrical Engineering，2021，90：1-20.
[61] REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：unified，real-time object detection[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：779-788.
[62] NERSISSON R，IYER T J，JOSEPH RAJ A N，et al.A dermoscopic skin lesion classification technique using YOLO-CNN and traditional feature model[J].Arabian Journal for Science and Engineering，2021，46（10）：9797-9808.
[63] LIU Z，XIONG R，JIANG T.Clinical-inspired network for skin lesion recognition[C]//23rd International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2020：340-350.
[64] 赵宸，帅仁俊，马力，等.基于Self-Attention-StyleGAN的皮肤癌图像生成与分类[J/OL].计算机工程与应用[2021-12-20].http：//kns.cnki.net/kcms/detail/11.2127.tp.20210419.
1440.067.html.
ZHAO C，SHUAI R J，MA L，et al.Skin cancer image generation and classification based on Self-Attention-StyleGAN[J/OL].Computer Engineering and Applications[2021-12-20].http：//kns.cnki.net/kcms/detail/11.2127.tp.20210419.
1440.067.html.
[65] GAO Y，TANG Z，ZHOU M，et al.Enabling data diversity：efficient automatic augmentation via regularized adversarial training[C]//27th International Conference on Information Processing in Medical Imaging.Cham：Springer，2021：85-97.
[66] LEI W，ZHANG R，YANG Y，et al.Class-center involved triplet loss for skin disease classification on imbalanced data[C]//2020 IEEE 17th International Symposium on Biomedical Imaging，2020：1-5.
[67] BARATA C，SANTIAGO C.Improving the explainability of skin cancer diagnosis using CBIR[C]//24th International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2021：550-559.
[68] ZHU W，LIAO H，LI W，et al.Alleviating the incompatibility between cross entropy loss and episode training for few-shot skin disease classification[C]//23rd International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2020：330-339.
[69] TOAAR M，CMERT Z，ERGEN B.Intelligent skin cancer detection applying autoencoder，MobileNetV2 and spiking neural networks[J].Chaos Solitons & Fractals，2021，144：110714.
[70] SANDLER M，HOWARD A，ZHU M，et al.MobileNetV2：inverted residuals and linear bottlenecks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2018：4510-4520.
[71] SRINIVASU P N，SIVASAI J G，IJAZ M F，et al.Classification of skin disease using deep learning neural networks with MobileNet V2 and LSTM[J].Sensors，2021，21（8）：2852.
[72] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[73] IQBAL I，YOUNUSB M，WALAYAT K，et al.Automated multi-class classification of skin lesions through deep convolutional neural network with dermoscopic images[J].Computerized Medical Imaging and Graphics，2020，88：101843.
[74] DORJ U O，LEE K K，CHOI J Y，et al.The skin cancer classification using deep convolutional neural network[J].Multimedia Tools and Applications，2018，77（8）：9909-9924.
[75] MINAGAWA A，KOGA H，SANO T，et al.Dermoscopic diagnostic performance of Japanese dermatologists for skin tumors differs by patient origin：a deep learning convolutional neural network closes the gap[J].The Journal of Dermatology，2021，48（2）：232-236.
[76] TYAGI A，MEHRA R.An optimized CNN based intelligent prognostics model for disease prediction and classification from dermoscopy images[J].Multimedia Tools and Applications，2020，79（35）：26817-26835.
[77] KHAN M A，SHARIF M，AKRAM T，et al.Skin lesion segmentation and multiclass classification using deep learning features and improved moth flame optimization[J].Diagnostics，2021，11（5）：811.
[78] ADLA D，REDDY G，NAYAK P，et al.Deep learning-based computer aided diagnosis model for skin cancer detection and classification[J].Distributed and Parallel Databases，2021.DOI：10.1007/s10619-021-07360-z.
[79] WANG D，PANG N，WANG Y Y，et al.Unlabeled skin lesion classification by self-supervised topology clustering network[J].Biomedical Signal Processing and Control，2021，66：102428.
[80] YOON C，HAMARNEH G，GARBI R.Generalizable feature learning in the presence of data bias and domain class imbalance with application to skin lesion classification[C]//22nd International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2019：365-373.
[81] BAYASI N，HAMARNEH G，GARBI R.Culprit-Prune-Net：efficient continual sequential multi-domain learning with application to skin lesion classification[C]//24th International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2021：165-175.
[82] AKROUT M，FARAHMAND A，JARMAIN T，et al.Improving skin condition classification with a visual symptom checker trained using reinforcement learning[C]//22nd International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2019：549-557.
[83] BDAIR T，NAVAB N，ALBARQOUNI S.FedPerl：semi-supervised peer learning for skin lesion classification[C]//24th International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2021：336-346.
[84] VAN MOLLE P，DE BOOM C，VERBELEN T，et al.Data-efficient sensor upgrade path using knowledge distillation[J].Sensors，2021，21（19）：6523.
[85] SHAMSHAD F，KHAN S，ZAMIR S W，et al.Transformers in medical imaging：a survey[J].arXiv：2201.09873，2022.
[86] WU W，MEHTA S，NOFALLAH S，et al.Scale-aware transformers for diagnosing melanocytic lesions[J].IEEE Access，2021，9：163526-163541.
[87] KINYANJUI N M，ODONGA T，CINTAS C，et al.Fairness of classifiers across skin tones in dermatology[C]//23rd International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2020：320-329.
[88] PACHECO A G C，KROHLING R.An attention-based mechanism to combine images and metadata in deep learning models applied to skin cancer classification[J].IEEE Journal of Biomedical and Health Informatics，2021，25（9）：3554-3563.