深度学习检测肺结节难点问题综述

doi:10.3778/j.issn.1002-8331.2402-0070

摘要/Abstract

摘要： 肺癌在全球范围内是致命性最高的癌症之一，肺结节是肺癌的早期表现形式，基于深度学习的肺结节检测模型因较高的检测准确率与效率逐渐成为辅助医生检测肺结节的有效方法。但是目前基于深度学习的肺结节检测模型仍有不足，一些重难点问题需要解决。第一，基于迁移学习、GAN网络、半监督学习与无监督学习解决模型训练时肺结节数据不足与类别不平衡问题；第二，增强模型特征提取能力提升对肺结节检测的敏感度与准确度；第三，提升模型假阳性肺结节筛查能力降低假阳性率；第四，加强模型检测肺结节的可解释能力；第五，基于大模型技术解决以上4个难点问题。最后，介绍检测模型训练与测试所需的数据集与评价指标并对未来肺结节检测优化方向进行讨论。

关键词: 肺癌, 肺结节检测, 深度学习, 计算机辅助检测系统

Abstract: Lung cancer is one of the deadliest cancers worldwide, and pulmonary nodules are an early manifestation of lung cancer. Detection models based on deep learning have gradually become an effective method to assist doctors in detecting pulmonary nodules due to their high detection accuracy and efficiency. However, current deep learning-based pulmonary nodule detection models still have shortcomings, and there are several challenging issues that need to be resolved. First, it addresses the issue of insufficient pulmonary nodule data and class imbalance during model training through transfer learning, GAN networks, semi-supervised learning, and unsupervised learning. Second, it enhances the model’s feature extraction capabilities to improve the sensitivity and accuracy of pulmonary nodule detection. Third, it improves the model’s capability to screen out false-positive pulmonary nodules, thereby reducing the false-positive rate. Fourth, it strengthens the model’s interpretability in detecting pulmonary nodules. Fifth, it solves the above four challenging issues using large model technology. Finally, it introduces the datasets and evaluation metrics required for the training and testing of detection models and discusses future optimization directions for pulmonary nodule detection.

Key words: lung cancer, lung nodule detection, deep learning, computer-aided detection system

包强强, 唐思源, 谷宇. 深度学习检测肺结节难点问题综述[J]. 计算机工程与应用, 2024, 60(19): 18-31.

BAO Qiangqiang, TANG Siyuan, GU Yu. Review of Difficult Problems of Deep Learning to Detect Lung Nodules[J]. Computer Engineering and Applications, 2024, 60(19): 18-31.

参考文献

[1] SUNG H, FERLAY J, SIEGEL R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. CA: A Cancer Journal for Clinicians, 2021, 71(3): 209-249.
[2] JIN N, SLOANE P J. Evaluation of pure ground glass pul- monary nodule: a case report[J]. Journal of Community Hospital Internal Medicine Perspectives, 2014, 4(4): 24562.
[3] KARKI A, SHAH R, FEIN A. Multiple pulmonary nodules in malignancy[J]. Current Opinion in Pulmonary Medicine, 2017, 23(4): 285-289.
[4] 孙华聪, 彭延军, 郭燕飞, 等. 3D多尺度深度卷积神经网络肺结节检测[J]. 中国图象图形学报, 2021, 26(7): 1716-1725.
SUN H C, PENG Y J, GUO Y F, et al. 3D multi scale deep convolutional neural networks in pulmonary nodule detec- tion[J]. Journal of Image and Graphics, 2021, 26(7): 1716-1725.
[5] ARMATO III S G, MCLENNAN G, BIDAUT L, et al. The lung image database consortium(LIDC) and image data- base resource initiative (IDRI): a completed reference data-base of lung nodules on CT scans[J]. Medical Physics, 2011, 38(2): 915-931.
[6] SETIO A A A, TRAVERSO A, DE BEL T, et al. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge[J]. Medical Image Analysis, 2017, 42: 1-13.
[7] PAN S J, YANG Q A. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22(10): 1345-1359.
[8] 陈道争, 江倩. 基于卷积神经网络和迁移学习的肺结节检测[J]. 计算机工程与设计, 2021, 42(1): 240-247.
CHEN D Z, JIANG Q. Pulmonary nodule detection based on convolutional neural networks with transfer learning[J]. Computer Engineering and Design, 2021, 42(1):240-247.
[9] TANG S, YANG M, BAI J. Detection of pulmonary nodules based on a multiscale feature 3D U-Net convolutional neural network of transfer learning [J]. PLoS One, 2020, 15(8): e0235672.
[10] 张驰名, 王庆凤, 刘志勤, 等. 基于深度迁移学习的肺结节辅助诊断方法[J]. 计算机工程, 2020, 46(1): 271-278.
ZHANG C M, WANG Q F, LIU Z Q, et al. Pulmonary nodule auxiliary diagnosis method based on deep transfer learning[J]. Computer Engineering, 2020, 46(1): 271-278.
[11] 梁俊杰，韦舰晶，蒋正锋. 生成对抗网络GAN综述[J]. 计算机科学与探索, 2020, 14(1): 1-17.
LIANG J J, WEI J J, JIANG Z F. Generative adversarial networks GAN overview[J]. Journal of Frontiers of Com-puter Science and Technology, 2020, 14(1): 1-17.
[12] 李阳, 高轼奇. 基于数据增强及注意力机制的肺结节检测系统[J]. 北京邮电大学学报, 2022, 45(4): 25-30.
LI Y, GAO S Q. Lung nodule detection system based on data augmentation and attention mechanism[J]. Journal of Beijing University of Posts and Telecommunications, 2022, 45(4): 25-30.
[13] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networksfrom overfitting[J]. The Journal of Machine Learning Research, 2014, 15(1): 1929-1958.
[14] GHIASI G, LIN T Y, LE Q V. DropBlock: a regularization method for convolutional networks[C]//Advances in Neural Information Processing Systems, 2018, 31.
[15] ONISHI Y, TERAMOTO A, TSUJIMOTO M, et al. Multi-planar analysis for pulmonary nodule classification in CT images using deep convolutional neural network and generative adversarial networks[J]. International Journal of Computer Assisted Radiology and Surgery, 2020, 15(1): 173-178.
[16] SUTTER T, DAUNHAWER I, VOGT J. Multimodal gene-rative learning utilizing Jensen-Shannon-divergence[C]//Ad-vances in Neural Information Processing Systems, 2020, 33: 6100-6110.
[17] SU Y X, ZHAO S L, CHEN X X, et al. Parallel Wasserstein generative adversarial nets with multiple discriminators[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019. San Francisco: Morgan Kaufmann, 2019: 3483-3489.
[18] GAO C F, CLARK S, FURST J, et al. Augmenting LIDC dataset using 3D generative adversarial networks to improve lung nodule detection[C]//Proceedings of the Conference on Medical Imaging Computer Aided Diagnosis, San Diego,?CA, Feb 17-20, 2019. Washington: SPIEW, 2019: 398-407.
[19] MESCHEDER L, GEIGER A, NOWOZIN S. Which train-ing methods for GANs do actually converge?[C]//Proceedings of the 35th International Conference on Machine Learning (ICML), Stockholm, Jul 10-15, 2018. New York: PMLR, 2018: 3481-3490.
[20] BU T, YANG Z Y, JIANG S, et al. 3D conditional generative adversarial network-based synthetic medical image aug- mentation for lung nodule detection[J]. International Jour-nal of Imaging Systems and Technology,2021, 31(2): 670-681.
[21] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Seattle, WA, Jun 27-30, 2016. Piscataway, NJ: IEEE Computer Society, 2016: 770-778.
[22] 肖鹏程, 徐文广, 张妍, 等. 基于SE注意力机制的废钢分类评级方法[J]. 工程科学学报, 2023, 45(8): 1342-1352.
XIAO P C, XU W G, ZHANG Y, et al. Research on scrap classification and rating method based on SE attention mechanism[J]. Chinese Journal of Engineering, 2023, 45(8): 1342-1352.
[23] VAN ENGELEN J E, HOOS H H. A survey on semi-supervised learning[J]. Machine Learning, 2020, 109(2):373-440.
[24] TANG S Y, MA R, LI Q Q, et al. Classification of benign and malignant pulmonary nodules based on the multire- solution 3D DPSECN model and semisupervised clustering[J]. IEEE Access, 2021, 9: 43397-43410.
[25] PANDEY A C, KULHARI A, SHUKLA D S. Enhancing sentiment analysis using roulette wheel selection based cuckoo search clustering method[J]. Journal of Ambient Intelligence and Humanized Computing, 2022, 13(1): 1-29.
[26] NALAIE K, GHIASI-SHIRAZI K, AKBARZADEH-T M R. Efficient implementation of a generalized convolutional neural networks based on weighted euclidean distance[C]//Proceedings of the 7th International Conference on Computer and Knowledge Engineering (ICCKE), Mashhad, Oct 26-27, 2017. Piscataway, NJ: IEEE Computer Society, 2017.
[27] SHAK K, AL-SHABI M, LIEW A, et al. A new semi- supervised self-training method for lung cancer prediction[J].?arXiv:2012.09472, 2020.
[28] WU Y H, PANG Y X, CAO P, et al. A 3D multi-scale virtual adversarial network for false positive reduction in pulmonary nodule detection[C]//Proceedings of the 3rd International Conference on Innovation in Artificial Intelligence (ICIAI), Suzhou, China, Mar 15-18, 2019. New York: ACM, 2019: 193-197.
[29] CHEN L, RUAN W, LIU X, et al. SeqVAT: virtual adversarial training for semi-supervised sequence labeling[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 8801-8811.
[30] DIKE H U, ZHOU Y, DEVEERASETTY K K, et al. Unsu-pervised learning based on artificial neural network: a review[C]//Proceedings of the IEEE International Conference on Cyborg and Bionic Systems (CBS), Shenzhen, China, Oct 25-27, 2018. Piscataway, NJ: IEEE Computer Society, 2018: 322-327.
[31] NIU C, WANG G. Unsupervised contrastive learning based transformer for lung nodule detection[J]. Physics in Medicine & Biology, 2022, 67(20): 204001.
[32] REDMON J, FARHADI A. YOLOv3: an incremental improve- ment[J]. arXiv:1804.02767, 2018.
[33] 郭晓敏, 黄新. 改进YOLOv3算法在肺结节检测中的应用[J]. 激光杂志, 2022, 43(5): 207-213.
GUO X M, HUANG X. Application of improved YOLOv3 algorithm in lung nodule detection[J]. Laser Journal, 2022, 43(5): 207-213.
[34] MISRA D. Mish: a self regularized nonmonotonic activation function[J]. arXiv:1908.08681, 2019.
[35] KAPOOR A, SINGHAL A. A comparative study of k-means, k-means++ and fuzzy c-means clustering algorithms[C]//Proceedings of the 3rd IEEE International Conference on Computational Intelligence and Communication Technology (CICT), Ghaziabad, India, Feb 9-10, 2017. Piscataway, NJ: IEEE Computer Society, 2017: 1-6.
[36] HOU Z, LIU X, CHEN L. Object detection algorithm for improving nonmaximum suppression using GIoU[C]//Proceedings of the 2019 2nd International Conference on Communication, Network and Artificial Intelligence, Guangzhou, China, Dec 27-29, 2019. Bristol, England: IOP Conference Series, 2020.
[37] 赵奎, 仇慧琪, 李旭, 等. 结合注意力和多路径融合的实时肺结节检测算法[J]. 计算机应用, 2024, 44(3): 945-952.
ZHAO K, QIU H Q, LI X, et al. Realtime pulmonary nodule detection algorithm combining attention and multipath fusion[J]. Journal of Computer Applications, 2024, 44(3): 945-952.
[38] JIANG P, ERGU D, LIU F, et al. A review of YOLO algorithm developments[J]. Procedia Computer Science, 2022, 199: 1066-1073.
[39] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, Sep 8-14, 2018. Berlin, Germany: Springer-Verlag, 2018: 3-19.
[40] WU X, ZHANG H, SUN J, et al. YOLO-MSRF for lung nodule detection[J]. Biomedical Signal Processing and Control, 2024, 94: 106318.
[41] WANG C Y, BOCHKOVSKIY A, LIAO H Y M. YOLOv7: trainable bag of freebies sets new state of the art for real-time object detectors[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, Jun 17-24, 2023. Los Alamitos, CA, USA: IEEE Computer Society, 2023: 7464-7475.
[42] 胥阳, 佘青山, 杨勇, 等. 基于密集残差连接的肺结节检测方法[J]. 传感技术学报, 2024, 37(1): 71-79.
XU Y, SHE Q S, YANG Y, et al. Lung nodule detection method based on dense residual connection[J]. Chinese Journal of Sensors and Actuators, 2024, 37(1): 71-79.
[43] CICEK O, ABDULKADIR A, LIENKAMP S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//Proceedings of the 19th International Conference on Medical Image Computing and Computer Assisted Intervention, Athens, Greece, Oct 17-21, 2016. Berlin, Germany: Springer-Verlag, 2016: 424-432.
[44] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, Jul 21-26, 2017. Piscataway, NJ: IEEE Computer Society, 2017: 4700-4708.
[45] 秦源源, 张鸿. 基于注意力特征金字塔网络的肺结节检测算法[J]. 计算机应用, 2023, 43(7): 2311-2318.
QIN Y Y, ZHANG H. Pulmonary nodule detection algorithm based on attention feature pyramid networks[J]. Journal of Computer Applications, 2023, 43(7): 2311-2318.
[46] ZHU L, ZHU H, YANG S, et al. Pulmonary nodule detection based on hierarchical-split HRNet and feature pyramid network with atrous convolution[J]. Biomedical Signal Processing and Control, 2023, 85: 105024.
[47] YUAN P, LIN S, CUI C, et al. HS-ResNet: hierarchical-split block on convolutional neural network[J]. arXiv: 2010.07621, 2020.
[48] ZHANG L, ZHANG J, LI Z, et al. A multiple channel and atrous convolution network for ultrasound image segme- ntation[J]. Medical Physics, 2020, 47(12): 6270-6285.
[49] CUI F, LI Y, LUO H, et al. SF2T: leveraging swin transfor-mer and two-stream networks for lung nodule detection[J]. Biomedical Signal Processing and Control, 2024, 95: 106389.
[50] LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, Oct 11-17, 2021. Piscataway, NJ: IEEE Computer Society, 2021: 10012-10022.
[51] 刘涌涛, 王宝珠, 郭志涛. 基于改进YOLOv7网络模型的肺结节检测算法[J]. 中国医学物理学杂志, 2023, 40(12): 1509-1517.
LIU Y T, WANG B Z, GUO Z T. Lung nodule detection algorithm using improved YOLOv7 network model[J]. Chinese Journal of Medical Physics, 2023, 40(12): 1509-1517.
[52] GEVORGYAN Z. SIoU loss: more powerful learning for bounding box regression[J]. arXiv:2205.12740, 2022.
[53] 王梦南, 赵涓涓, 肖宁, 等. 基于可变形卷积神经网络的肺结节假阳性识别[J]. 计算机工程与设计, 2022, 43(6): 1732-1739.
WANG M N, ZHAO J J, XIAO N, et al. False positive identification of pulmonary nodules based on deformable convolutional neural network[J]. Computer Engineering and Design, 2022, 43(6): 1732-1739.
[54] ZUO W X, ZHOU F Q, HE Y Z. An embedded multi-branch 3D convolution neural network for false positive reduction in lung nodule detection[J]. Journal of Digital Imaging, 2020, 33(4): 846-857.
[55] ZHAO D , LIU Y, YIN H , et al. A novel multi scale CNNs for false positive reduction in pulmonary nodule detection[J]. Expert Systems with Applications, 2022, 207: 117652.
[56] 张佳嘉, 张小洪. 多分支卷积神经网络肺结节分类方法及其可解释性[J]. 计算机科学, 2020, 47(9): 129-134. ZHANG J J, ZHANG X H. Multi-branch convolutional neural network for lung nodule classification and its interpretability[J]. Computer Science, 2020, 47(9): 129-134.
[57] SHEN S, HAN S X, ABERLE D R, et al. An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification[J]. Expert Systems with Applications, 2019, 128: 84-95.
[58] 罗锦钊, 孙玉龙, 钱增志, 等.人工智能大模型综述及展望[J]. 无线电工程, 2023, 53(11): 2461-2472.
LUO J Z, SUN Y L, QIAN Z Z, et al. Overview and prospect of artificial intelligence large models[J]. Radio Engineering, 2023, 53(11): 2461-2472.
[59] ZHONG R, XU Y, ZHANG C, et al. Leveraging large language model to generate a novel metaheuristic algorithm with crispe framework[J]. arXiv:2403.16417, 2024.
[60] DOI H, OSAWA T, TSUTSUMIDA N. The role of large pre-trained models in ecology and biodiversity conservation: opportunities and challenges[J]. Authorea Preprints, 2024.
[61] GU J, CHO H C, KIM J, et al. CheX-GPT: harnessing large language models for enhanced chest X-ray report labeling[J]. arXiv:2401.11505, 2024.
[62] SMIT A, JAIN S, RAJPURKAR P, et al. CheXbert: combi-ning automatic labelers and expert annotations for accurate radiology report labeling using BERT[J]. arXiv:2004.09167, 2020.
[63] RADFORD A, KIM J W, HALLACY C, et al. Learning transferable visual models from natural language superv- ision[C]//Proceedings of the International Conference on Machine Learning, San Diego, CA, United States, Jul 18-24, 2021. New York: ACM, 2021: 8748-8763.
[64] LEI Y, LI Z, SHEN Y, et al. CLIP-lung: textual knowledge guided lung nodule malignancy prediction[C]//Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention, Vancouver, Canada, Oct 8-12, 2023. Cham: Springer-Verlag, 2023: 403-412.
[65] LUO Y, HOOSHANGNEJAD H, FENG X, et al. False posi- tive reduction in pulmonary cancer detection based on GPT-4V[C]//Proceedings of the ?International Conference on Medical Imaging with Deep Learning, Paris, France, Jul 3, 2024.
[66] YANG Z, LI L, LIN K, et al. The dawn of LMMs: preliminary explorations with GPT-4V (ision)[J]. arXiv:2309.17421, 2023.
[67] HOOSHANGNEJAD H, FENG X, HUANG G, et al. EXACT-Net: EHR-guided lung tumor auto-segmentation for non-small cell lung cancer radiotherapy[J]. arXiv:2402. 14099, 2024.
[68] SHAHIN M, CHEN F F, HOSSEINZADEH A, et al. A novel approach to voice of customer extraction using GPT-3.5 Turbo: linking advanced NLP and lean six sigma4.0[J]. The International Journal of Advanced Manufacturing Technology, 2024: 1-16.
[69] BRAGAGNOLO A, TARTAGLIONE E, FIANDROTTI A, et al. ON the role of structured pruning for neural network compression[C]//Proceedings of the IEEE International Conference on Image Processing (ICIP), Sep 19-22, 2021. Piscataway, NJ: IEEE Computer Society, 2021.
[70] 邵仁荣, 刘宇昂, 张伟, 等. 深度学习中知识蒸馏研究综述[J]. 计算机学报, 2022, 45(8): 1638-1673.
SHAO R R, LIU Y A, ZHANG W, et al. A survey of knowledge distillation in deep learning[J]. Chinese Journal of Computers, 2022, 45(8): 1638-1673.
[71] JACOB B, KLIGYS S, CHEN B, et al. Quantization and training of neural networks for efficient integer-arithmetic-only inference[C]//Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, Jun 18-23, 2018. Piscataway,NJ: IEEE Computer Society, 2018: 2704-2713.
[72] ZHANG X, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural networkfor mobile devices[C]//Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, Jun 18-23, 2018. Piscataway, NJ: IEEE Computer Society, 2018: 6848-6856.