Application of Deep Learning in Auxiliary Diagnosis of Pulmonary Nodules

doi:10.3778/j.issn.1002-8331.2201-0397

Abstract

Abstract: Lung cancer ranks first in cancer mortality, and its early diagnosis and treatment can reduce the mortality rate of lung cancer patients. Deep learning can automatically extract nodule features, and complete the classification of benign, malignant and malignant grades of pulmonary nodules. Therefore, deep learning methods have become an important means of early diagnosis of lung cancer. This paper introduces the commomly used datasets, then systematically expounds the application of stacked denoising autoencoder（SDAE）, deep belief network（DBN）, generative adversarial network（GAN）, convolutional neural network（CNN）, recurrent neural network（RNN） and transfer learning techniques in the classification of benign and malignant pulmonary nodules. And then, the application of deep convolutional generative adversarial network（DCGAN）, multi-scale convolutional neural network（MCNN）, U-shaped network（U-Net） and ensemble learning techniques in the classification of malignant grades of pulmonary nodules is described. A comprehensive analysis of deep learning methods for pulmonary nodules is carried out, and the future research directions are prospected.

Key words: pulmonary nodules, deep learning, benign and malignant classification, malignant grade classification, computer-aided diagnosis

摘要： 肺癌位居癌症死亡率首位，对其进行早期诊断和治疗可降低肺癌患者的死亡率。深度学习能够自动提取结节特征，并完成肺结节的良恶性及恶性等级分类，因此深度学习方法成为肺癌早期诊断的重要手段。对常用数据集进行介绍，系统阐述了栈式去噪自编码器（SDAE）、深度置信网络（DBN）、生成对抗网络（GAN）、卷积神经网络（CNN）、循环神经网络（RNN）和迁移学习技术在肺结节良恶性分类中的应用，阐述了深度卷积生成对抗网络（DCGAN）、多尺度卷积神经网络（MCNN）、U型网络（U-Net）和集成学习技术在肺结节恶性等级分类中的应用，针对肺结节分类的深度学习方法进行了综合分析，并对未来研究方向进行展望。

关键词: 肺结节, 深度学习, 良恶性分类, 恶性等级分类, 计算机辅助诊断

FENG Yanyan, WEI Dejian, NI Wei. Application of Deep Learning in Auxiliary Diagnosis of Pulmonary Nodules[J]. Computer Engineering and Applications, 2022, 58(18): 59-70.

冯妍妍, 魏德健, 倪伟. 深度学习在肺结节辅助诊断中的应用[J]. 计算机工程与应用, 2022, 58(18): 59-70.

References

[1] SUNG H，FERLAY J，SIEGEL R L，et al.Global cancer statistics 2020：globocan estimates of incidenceand mortality worldwide for 36 cancers in 185 countries[J].Ca：a Cancer Journal for Clinicians，2021，71（3）：209-249.
[2] QIU H，CAO S，XU R.Cancer incidence，mortality，and burden in China：a time-trend analysis and comparison with the United States and United Kingdom based on the global epidemiological data released in 2020[J].Cancer Communications，2021，41（10）：1037-1048.
[3] CAREY S，KANDEL S，FARRELL C，et al.Comparison of conventional chest X ray with a novel projection technique for ultra-low dose CT[J].Medical Physics，2021，48（6）：2809-2815.
[4] WANG Y，WANG J，YANG S，et al.Proposing a deep learning-based method for improving the diagnostic certainty of pulmonary nodules in CT scan of chest[J].European Radiology，2021，31（11）：8160-8167.
[5] 张福玲，张少敏.应用于CT图像肺结节检测的深度学习方法综述[J].计算机工程与应用，2020，56（13）：20-32.
ZHANG F L，ZHANG S M.Review of deep learning methods applied to lung nodule detection in CT images[J].Computer Engineering and Applications，2020，56（13）：20-32.
[6] LI D，MIKELA VILMUN B，FREDERIK CARLSEN J，et al.The performance of deep learning algorithms on automatic pulmonary nodule detection and classification tested on different datasets that are not derived from LIDC-IDRI：a systematic review[J].Diagnostics，2019，9（4）：207.
[7] 王远航，唐震.神经网络在CT肺结节的分类应用进展[J].牡丹江医学院学报，2021，42（2）：135-137.
WANG Y H，TANG Z.Advances in the classification and application of neural network in CT pulmonary nodules[J].Journal of Mudanjiang Medical University，2021，42（2）：135-137.
[8] JIANG W，ZENG G，WANG S，et al.Application of deep learning in lung cancer imaging diagnosis[J].Journal of Healthcare Engineering，2022，52（5）：584-596.
[9] FARHAT H，SAKR G E，KILANY R.Deep learning applications in pulmonary medical imaging：recent updates and insights on Covid-19[J].Machine Vision and Applications，2020，31（6）：1-42.
[10] FRATTINI M，FROESCH P，EPISTOLIO S.Overview of recent advances in molecular analysis for diagnosing early stage lung cancer nodules[J].Translational Lung Cancer Research，2021，10（11）：4303.
[11] GUERRINI S，DEL ROSCIO D，ZANONI M，et al.Lung cancer imaging：screening result and nodule management[J].International Journal of Environmental Research and Public Health，2022，19（4）：2460.
[12] ARMATO III S G，MCLENNAN G，BIDAUT L，et al.The lung image database consortium（LIDC） and image database resource initiative（IDRI）：a completed reference database of lung nodules on CT scans[J].Medical Physics，2011，38（2）：915-931.
[13] SETIO A A A，TRAVERSO A，DE BEL T，et al.Validation，comparison，and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images：the luna16 challenge[J].Medical Image Analysis，2017，42：1-13.
[14] Trial summary-learn-NLST—the cancer data access system[EB/OL].[2019-12-31].https：//biometry.nci.nih.gov/cdas/
learn/nlst/trial-summary/.
[15] KUAN K，RAVAUT M，MANEK G，et al.Deep learning for lung cancer detection：tackling the kaggle data science bowl 2017 challenge[J].arXiv：1705.09435，2017.
[16] SHIRAISHI J，KATSURAGAWA S，IKEZOE J，et al.Development of a digital image database for chest radiographs with and without a lung nodule[J].American Journal of Roentgenology，2000，174（1）：71-74.
[17] Danish lung cancer screening trial（DLCST）.Full text view-clinicaltrials[EB/OL].[2019-12-31].https：//clinicaltrials.gov/ct2/show/NCT00496977.
[18] Alitianchi data[EB/OL].[2019-12-31].https：//tianchi.ali-yun.com/competition/entrance/231601/information.
[19] ARMATO III，SAMUEL G，HADJIISKI L，et al.SPIE-AAPM-NCI lung nodule classification challenge dataset[C]//Proceedings of 2015 SPIE Medieal Imaging Conference，2015.
[20] 孙浩天，袁刚，杨杨，等.基于三维各向异性卷积的肺结节良恶性分类[J].计算机工程与应用，2021，57（10）：133-138.
SUN H T，YUAN G，YANG Y，et al.3D anisotropic convolution based pulmonary nodule classification[J].Computer Engineering and Applications，2021，57（10）：133-138.
[21] LU X，GU Y，YANG L，et al.Multi-level 3d densenets for false-positive reduction in lung nodule detection based on chest computed tomography[J].Current Medical Imaging，2020，16（8）：1004-1021.
[22] 罗嘉滢，赵涓涓，强彦，等.基于多特征广义深度自编码的肺结节诊断方法[J].计算机工程与设计，2019，40（1）：154-160.
LUO J Y，ZHAO J J，QIANG Y，et al.Diagnosis of pulmonary nodules based on multi-feature generalized depth autoencoding[J].Computer Engineering and Design，2019，40（1）：154-160.
[23] 张华丽，康晓东，冉华，等.用于肺结节影像分类识别的DBN与CNN的比较研究[J].计算机科学，2020，47（S1）：254-259.
ZHANG H L，KANG X D，RAN H，et al.A comparative study of DBN and CNN for image classification and recognition of lung nodules[J].Computer Science，2020，47（S1）：254-259.
[24] 张婷，赵文婷，赵涓涓，等.改进的深度信念网络肺结节良恶性分类[J].计算机工程与设计，2018，39（9）：2707-2713.
ZHANG T，ZHAO W T，ZHAO J J，et al.Improved deep belief network classification of benign and malignant pulmonary nodules[J].Computer Engineering and Design，2018，39（9）：2707-2713.
[25] LI Y，LIAO C，WANG Y，et al.Energy-efficient optimal relay selection in cooperative cellular networks based on double auction[J].IEEE Transactions on Wireless Communications，2015，14（8）：4093-4104.
[26] HSU S，YANG C，HUANG C，et al.Semistargan：semi-supervised generative adversarial networks for multi-domain image-to-image translation[C]//Asian Conference on Computer Vision，2018：338-353.
[27] KUANG Y，LAN T，PENG X，et al.Unsupervised multi-discriminator generative adversarial network for lung nodule malignancy classification[J].IEEE Access，2020，8：77725-77734.
[28] 李莉，张浩洋，乔璐.基于改进深度卷积对抗生成网络的肺结节良恶性分类[J].计算机工程，2020，46（12）：262-269.
LI L，ZHANG H Y，QIAO L.Benign and malignant classification of lung nodules based on improved deep convolutional adversarial generation network[J].Computer Engineering，2020，46（12）：262-269.
[29] HUA K，HSU C，HIDAYATI S C，et al.Computer-aided classification of lung nodules on computed tomography images via deep learning technique[J].Oncotargets and Therapy，2015，8.
[30] SHEN W，ZHOU M，YANG F，et al.Multi-crop convolutional neural networks for lung nodule malignancy suspiciousness classification[J].Pattern Recognition，2017，61：663-673.
[31] XIE Y，XIA Y，ZHANG J，et al.Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest CT[J].IEEE Transactions on Medical Imaging，2019，38（4）：991-1004.
[32] ZUO W，ZHOU F，LI Z，et al.Multi-resolution CNN and knowledge transfer for candidate classification in lung nodule detection[J].IEEE Access，2019，7：32510-32521.
[33] XIE S，TU Z.Holistically-nested edge detection[C]//2015 IEEE International Conference on Computer Vision（ICCV），2016.
[34] LEI Y，TIAN Y，SHAN H，et al.Shape and margin-aware lung nodule classification in low-dose CT images via soft activation mapping[J].Medical Image Analysis，2020，60：101628.
[35] LIN T，DOLLáR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2117-2125.
[36] SHEN L，WANG X，GAO M，et al.Classification of benign-malignant pulmonary nodules based on multi-view improved dense network[C].[S.l.]：Springer International Publishing，2021：582-593.
[37] VEASEY B，FARHANGI M M，FRIGUI H，et al.Lung nodule malignancy classification based on nlstx data[C]//2020 IEEE 17th International Symposium on Biomedical Imaging（ISBI），2020：1870-1874.
[38] VEASEY B P，BROADHEAD J，DAHLE M，et al.Lung nodule malignancy prediction from longitudinal CT scans with siamese convolutional attention networks[J].IEEE Open Journal of Engineering in Medicine and Biology，2020，1：257-264.
[39] LIMA T J B，ARAIUJO F H D D，FILHO A O D C，et al.Evaluation of data balancing techniques in 3D CNNs for the classification of pulmonary nodules in CT images[C]//2020 IEEE Symposium on Computers and Communications（ISCC），2020.
[40] 杨建利，朱德江，邵嘉俊，等.三维多尺度交叉融合网络肺结节分类研究[J].计算机工程与应用，2022，58（14）：121-125.
YANG J L，ZHU D J，SHAO J J，et al.Research on classification of pulmonary nodules by three-dimensional multi-scale cross fusion network[J].Computer Engineering and Applications，2022，58（14）：121-125.
[41] LV W，WANG Y，ZHOU C，et al.Development and validation of a clinically applicable deep learning strategy（honors） for pulmonary nodule classification at CT：a retrospective multicentre study[J].Lung Cancer，2021，155：78-86.
[42] AL-SHABI M，LAN B L，CHAN W Y，et al.Lung nodule classification using deep local-global networks[J].International Journal of Computer Assisted Radiology and Surgery，2019，14（10）：1815-1819.
[43] LEI Y，TIAN Y，SHAN H，et al.Shape and margin-aware lung nodule classification in low-dose CT images via soft activation mapping[J].Medical Image Analysis，2019，60：101628.
[44] CHEN Y，LI J，XIAO H，et al.Dual path networks[J].arXiv：1707.01629，2017.
[45] JIANG H，GAO F，XU X，et al.Attentive and ensemble 3d dual path networks for pulmonary nodules classification[J].Neurocomputing，2020，398：422-430.
[46] TANG S，MA R，LI Q，et al.Classification of benign and malignant pulmonary nodules based on the multiresolution 3d dpsecn model and semisupervised clust[J].IEEE Access，2021（99）.
[47] XIA K，CHI J，GAO Y，et al.Adaptive aggregated attention network for pulmonary nodule classification[J].Applied Sciences，2021，11（2）：610.
[48] POLAT H，DANAEI MEHR H.Classification of pulmonary CT images by using hybrid 3d-deep convolutional neural network architecture[J].Applied Sciences，2019，9（5）：940.
[49] RAFAEL-PALOU X，AUBANELL A，BONAVITA I，et al.Pulmonary nodule malignancy classification using its temporal evolution with two-stream 3d convolutional neural networks[J].arXiv：2005.11341，2020.
[50] 杨帆，谢红薇，刘爱媛.基于卷积神经网络的肺结节分类算法[J].计算机工程与应用，2019，55（7）：145-150.
YANG F，XIE H W，LIU A Y.Lung nodules classification algorithm based on convolutional neural network[J].Computer Engineering and Applications，2019，55（7）：145-150.
[51] ASTARAKI M，ZAKKO Y，DASU I T，et al.Benign-malignant pulmonary nodule classification in low-dose CT with convolutional features[J].Physica Medica，2021，83：146-153.
[52] ZHAO J，ZHANG C，LI D，et al.Combining multi-scale feature fusion with multi-attribute grading，a CNN model for benign and malignant classification of pulmonary nodules[J].Journal of Digital Imaging，2020，33（4）：869-878.
[53] NI Z，PENG Y.A serialized classification method for pulmonary nodules based on lightweight cascaded convolutional neural network-long short-term memory[J].International Journal of Imaging Systems and Technology，2020，30（4）：950-962.
[54] WANG C，CHEN D，HAO L，et al.Pulmonary image classification based on inception-v3 transfer learning model[J].IEEE Access，2019，7：146533-146541.
[55] HUANG C，LV W，ZHOU C，et al.Discrimination between transient and persistent subsolid pulmonary nodules on baseline CT using deep transfer learning[J].European Radiology，2020.
[56] APOSTOLOPOULOS I D，PINTELAS E G，LIVIERIS I E，et al.Automatic classification of solitary pulmonary nodules in PEt/Ct imaging employing transfer learning techniques[J].Medical & Biological Engineering，2021，59（7）.
[57] ZHANG G，YANG Z，GONG L，et al.Classification of benign and malignant lung nodules from CT images based on hybrid features[J].Physics in Medicine & Biology，2019，64（12）：125011.
[58] WANG W，CHAKRABORTY G，CHAKRABORTY B.3D multi-scale densenet for malignancy grade classification of pulmonary nodules[C]//2020 11th International Conference on Awareness Science and Technology，2020：1-4.
[59] CHEN H，XIA Y，DUAN H，et al.A computed tomography signs quantization analysis method for pulmonary nodules malignancy grading[J].International Journal of Imaging Systems and Technology，2021.
[60] MUZAMMIL M，ALI I，HAQ I U，et al.Pulmonary nodule classification using feature and ensemble learning-based fusion techniques[J].IEEE Access，2021，9：113415-113427.
[61] WANG D，ZHANG T，LI M，et al.3d deep learning based classification of pulmonary ground glass opacity nodules with automatic segmentation[J].Computerized Medical Imaging and Graphics，2021，88：101814.
[62] BONAVITA I，RAFAEL-PALOU X，CERESA M，et al.Integration of convolutional neural networks for pulmonary nodule malignancy assessment in a lung cancer classification pipeline[J].Computer Methods and Programs in Biomedicine，2020，185：105172.
[63] DA NóBREGA R V M，REBOU?AS FILHO P P，RODRIGUES M B，et al.Lung nodule malignancy classification in chest computed tomography images using transfer learning and convolutional neural networks[J].Neural Computing and Applications，2020，32（15）：11065-11082.
[64] AL-SHABI M，LEE H K，TAN M.Gated-dilated networks for lung nodule classification in CT scans[J].IEEE Access，2019，7：178827-178838.
[65] 徐久强，洪丽萍，朱宏博，等.一种用于肺结节恶性度分类的生成对抗网络[J].东北大学学报（自然科学版），2018，39（11）：1556-1561.
XU J Q，HONG L P，ZHU H B，et al.A generative adversarial network for malignancy classification of lung nodules[J].Journal of Northeastern University（Natural Science Edition），2018，39（11）：1556-1561.
[66] MESSAY T，HARDIE R C，TUINSTRA T R.Segmentation of pulmonary nodules in computed tomography using a regression neural network approach and its application to the lung image database consortium and image database resource initiative dataset[J].Medical Image Analysis，2015，22（1）：48-62.
[67] RONNEBERGER O，FISCHER P，BROX T.U-net：convolutional networks for biomedical image segmentation[C].[S.l.]：Springer International Publishing，2015：234-241.
[68] LIAO F，LIANG M，LI Z，et al.Evaluate the malignancy of pulmonary nodules using the 3-d deep leaky noisy-or network[J].IEEE Transactions on Neural Networks and Learning Systems，2019，30（11）：3484-3495.
[69] XIAO N，QIANG Y，BILAL ZIA M，et al.Ensemble classification for predicting the malignancy level of pulmonary nodules on chest computed tomography images[J].Oncology Letters，2020，20（1）：401-408.