深度学习在骨质疏松辅助诊断中的应用

doi:10.3778/j.issn.1002-8331.2305-0030

摘要/Abstract

摘要： 骨质疏松症是一种由于骨密度下降引起骨折危险性增加的全身性疾病，临床上以影像学检查作为诊断依据。近几年深度学习方法在骨骼医学图像处理领域取得突破性进展，针对骨质疏松辅助诊断中所采用的深度学习方法进行了梳理总结。介绍了常用的影像学数据集，系统阐述了卷积神经网络、循环神经网络、深度置信网络、生成对抗网络在骨质疏松分类中的应用，阐述了全卷积网络、U-Net在骨质疏松病灶区域分割中的应用，同时介绍了最新AI模型ChatGPT的潜在应用，比较不同模型的性能，指出该领域目前存在的难点并提出相应的展望。

关键词: 骨质疏松症, 深度学习, 计算机辅助诊断, 卷积神经网络

Abstract: Osteoporosis, a systemic disease with an increased risk of fracture, is caused by a decrease in bone density. In clinical practice, the diagnosis is based on the imaging examination. In recent years, deep learning methods have made breakthroughs in the field of skeletal medical image processing. This paper compares and summarizes the deep learning methods that have been used in osteoporosis assisted diagnosis. Firstly, it introduces the commonly used datasets. Secondly, it systematically expounds the application of convolutional neural network, recurrent neural network, deep belief network and generative adversarial network in the classification of osteoporosis. And then, the application of full convolutional networks and U-Net in the segmentation of the lesion area of osteoporosis is described. And it introduces the model of the latest AI ChatGPT potential applications. Then it compares the performance of the different models. Finally, the paper points out the existing difficulties and puts forward the corresponding prospects in this field.

Key words: osteoporosis, deep learning, computer aided diagnosis, convolutional neural network

姜良, 张程, 魏德健, 曹慧, 杜昱峥. 深度学习在骨质疏松辅助诊断中的应用[J]. 计算机工程与应用, 2024, 60(7): 26-40.

JIANG Liang, ZHANG Cheng, WEI Dejian, CAO Hui, DU Yuzheng. Deep Learning in Aided Diagnosis of Osteoporosis[J]. Computer Engineering and Applications, 2024, 60(7): 26-40.

参考文献

[1] COMPSTON J, COOPER A, COOPER C, et al. UK clinical guideline for the prevention and treatment of osteoporosis[J]. Archives of Osteoporosis, 2017, 12: 1-24.
[2] SI L, WINZENBERG T M, JIANG Q, et al. Projection of osteoporosis-related fractures and costs in China: 2010—2050[J]. Osteoporosis International, 2015, 26: 1929-1937.
[3] ADAMI G, FASSIO A, GATTI D, et al. Osteoporosis in 10 years time: a glimpse into the future of osteoporosis[J]. Therapeutic Advances in Musculoskeletal Disease, 2022, 14: 1759720X221083541.
[4] 史哲, 万全增, 李春雯. 医学影像学在骨质疏松症诊断中的应用价值[J]. 中国骨质疏松杂志, 2010, 16(12): 969-971.
SHI Z, WAN Q Z, LI C W. Potential of medical imaging in osteoporosis diagnosis[J]. Chinese Journal of Osteoporosis, 2010, 16(12): 969-971.
[5] 田伟, 韩晓光, 张琦. 人工智能在骨科中的应用及展望[J]. 中华创伤骨科杂志, 2021, 23(4): 277-280.
TIAN W, HAN X G, ZHANG Q. Application and prospect of artificial intelligence in orthopedics[J]. Chinese Journal of Orthopaedic Trauma, 2021, 23(4): 277-280.
[6] KALMET P H S, SANDULEANU S, PRIMAKOV S, et al. Deep learning in fracture detection: a narrative review[J]. Acta Orthopaedica, 2020, 91(2): 215-220.
[7] FATHIMA S M N, TAMILSELVI R, BEHAM M P. DEXSIT: a benchmark database for BMD measurement and analysis[C]//2018 Fourth International Conference on Biosignals, Images and Instrumentation, 2018: 206-212.
[8] FATHIMA S M N, TAMILSELVI R, BEHAM M P. XSITRAY: a database for the detection of osteoporosis condition[J]. Biomedical and Pharmacology Journal, 2019, 12(1): 267-271.
[9] RAJPURKAR P, IRVIN J, BAGUL A, et al. Mura dataset: towards radiologist-level abnormality detection in muscu-loskeletal radiographs[J]. arXiv:1712.06957, 2017.
[10] CHENG P, YANG Y, YU H, et al. Automatic vertebrae locali-
zation and segmentation in CT with a two-stage Dense-U-Net[J]. Scientific Reports, 2021, 11(1): 22156.
[11] SEKUBOYINA A, HUSSEINI M E, BAYAT A, et al. VerSe: a vertebrae labelling and segmentation benchmark for multi-detector CT images[J]. Medical Image Analysis, 2021, 73: 102166.
[12] DENG Y, WANG C, HUI Y, et al. CTSpine1K: a large-scale dataset for spinal vertebrae segmentation in computed tomography[J]. arXiv:2105.14711, 2021.
[13] AREECKAL A S, KOCHER M. Current and emerging diagnostic imaging-based techniques for assessment of osteoporosis and fracture risk[J]. IEEE Reviews in Biomedical Engineering, 2018, 12: 254-268.
[14] 李小玲, 易银芝, 董林, 等. 人工智能技术在骨科领域的应用进展[J]. 全科护理, 2020, 18(29): 3937-3939.
LI X L, YI Y Z, DONG L, et al. Advances in the application of artificial intelligence technology in orthopedics[J]. Chinese General Practice Nursing, 2020, 18(29): 3937-3939.
[15] GUO G, WANG H, BELL D, et al. KNN model-based approach in classification[C]//OTM Confederated International Conferences “On the Move to Meaningful Internet Systems”, 2003: 986-996.
[16] HEARST M A, DUMAIS S T, OSUNA E, et al. Support vector machines[J]. IEEE Intelligent Systems and Their Applications, 1998, 13(4): 18-28.
[17] LAVALLEY M P. Logistic regression[J]. Circulation, 2008, 117(18): 2395-2399.
[18] 杨培伟, 周余红, 邢岗, 等. 卷积神经网络在生物医学图像上的应用进展[J]. 计算机工程与应用, 2021, 57(7): 44-58.
YANG P W, ZHOU Y H, XING G, et al. Applications of convolutional neural network in biomedical image[J]. Computer Engineering and Applications, 2021, 57(7): 44-58.
[19] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[20] JANG R, CHOI J H, KIM N, et al. Prediction of osteoporosis from simple hip radiography using deep learning algorithm[J]. Scientific Reports, 2021, 11(1): 19997.
[21] WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7794-7803.
[22] SIMONYAN K, ZISSERMAN A. Very deep convolu-tional networks for large-scale image recognition[J]. arXiv:1409.1556,2014.
[23] LEE K S, JUNG S K, RYU J J, et al. Evaluation of transfer learning with deep convolutional neural networks for screening osteoporosis in dental panoramic radiographs[J]. Journal of Clinical Medicine, 2020, 9(2): 392.
[24] 杨莉, 万旺根. 基于预训练-微调策略的COVID-19预测模型[J]. 计算机工程, 2022, 48(3): 17-22.
YANG L, WAN W G. COVID-19 prediction model based on pre-training and fine-tuning strategy[J]. Computer Engineering, 2022, 48 (3): 17-22.
[25] 陈道争, 江倩. 基于卷积神经网络和迁移学习的肺结节检测[J]. 计算机工程与设计, 2021, 42(1): 240-247.
CHEN D Z, JIANG Q. Pulmonary nodule detection based on convolutional neural networks with transfer learning[J]. Computer Engineering and Design, 2021, 42(1): 240-247.
[26] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
[27] YAMAMOTO N, SUKEGAWA S, YAMASHITA K, et al. Effect of patient clinical variables in osteoporosis classification using hip X-rays in deep learning analysis[J]. Medicina, 2021, 57(8): 846.
[28] KONG S H, LEE J W, BAE B U, et al. Development of a spine X-ray-based fracture prediction model using a deep learning algorithm[J]. Endocrinology and Metabolism, 2022, 37(4): 674-683.
[29] LEE S, CHOE E K, KANG H Y, et al. The exploration of feature extraction and machine learning for predicting bone density from simple spine X-ray images in a Korean population[J]. Skeletal Radiology, 2020, 49: 613-618.
[30] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 2818-2826.
[31] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[32] SPEISER J L, MILLER M E, TOOZE J, et al. A comparison of random forest variable selection methods for classification prediction modeling[J]. Expert Systems with Applications, 2019, 134: 93-101.
[33] YAMAMOTO N, SUKEGAWA S, KITAMURA A, et al. Deep learning for osteoporosis classification using hip radiographs and patient clinical covariates[J]. Biomolecules, 2020, 10(11): 1534.
[34] TAN M, LE Q. Efficientnet: rethinking model scaling for convolutional neural networks[C]//International Conference on Machine Learning, 2019: 6105-6114.
[35] SUKEGAWA S, FUJIMURA A, TAGUCHI A, et al. Identification of osteoporosis using ensemble deep learning model with panoramic radiographs and clinical covariates[J]. Scientific Reports, 2022, 12(1): 1-10.
[36] SELVARAJU R R, Cogswell M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 618-626.
[37] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 4700-4708.
[38] MAO L, XIA Z, PAN L, et al. Deep learning for screening primary osteopenia and osteoporosis using spine radio-graphs and patient clinical covariates in a Chinese population[J]. Frontiers in Endocrinology, 2022, 13: 971877.
[39] XUE L, QIN G, CHANG S, et al. Osteoporosis prediction in lumbar spine X-ray images using the multi-scale weighted fusion contextual transformer network[J]. Artificial Intelligence in Medicine, 2023: 102639.
[40] LIU J, WANG J, RUAN W, et al. Diagnostic and gradation model of osteoporosis based on improved deep U-Net network[J]. Journal of Medical Systems, 2020, 44: 1-7.
[41] ZHANG B, YU K, NING Z, et al. Deep learning of lumbar spine X-ray for osteopenia and osteoporosis screening: a multicenter retrospective cohort study[J]. Bone, 2020, 140: 115561.
[42] NGUYEN T P, CHAE D S, PARK S J, et al. A novel approach for evaluating bone mineral density of hips based on Sobel gradient-based map of radiographs utilizing convolutional neural network[J]. Computers in Biology and Medicine, 2021, 132: 104298.
[43] LEULIET T, MAXIM V, PEYRIN F, et al. Impact of the training loss in deep learning-based CT reconstruction of bone microarchitecture[J]. Medical Physics, 2022, 49(5): 2952-2964.
[44] SATO Y, YAMAMOTO N, INAGAKI N, et al. Deep learning for bone mineral density and T-score prediction from chest X-rays: a multicenter study[J]. Biomedicines, 2022, 10(9): 2323.
[45] DONG Q, LUO G, LANE N E, et al. Deep learning classification of spinal osteoporotic compression frac-tures on radiographs using an adaptation of the genant semiquantitative criteria[J]. Academic Radiology, 2022, 29(12): 1819-1832.
[46] MU S, WANG J, GONG S. Application of medical imaging based on deep learning in the treatment of lumbar degenerative diseases and osteoporosis with bone cement screws[J]. Computational and Mathematical Methods in Medicine, 2021(15): 1-10.
[47] YASAKA K, AKAI H, KUNIMATSU A, et al. Prediction of bone mineral density from computed tomography: application of deep learning with a convolutional neural network[J]. European Radiology, 2020, 30: 3549-3557.
[48] HU X, ZHU Y, QIAN Y, et al. Prediction of subsequent osteoporotic vertebral compression fracture on CT radiography via deep learning[J]. View, 2022, 3(6): 20220012.
[49] TANG C, ZHANG W, LI H, et al. CNN-based qualitative detection of bone mineral density via diagnostic CT slices for osteoporosis screening[J]. Osteoporosis International, 2021, 32: 971-979.
[50] LIPTON Z C, BERKOWITZ J, ELKAN C. A critical review of recurrent neural networks for sequence learning[J]. arXiv:1506.00019, 2015.
[51] SHAKER A S. Detection and segmentation of osteoporosis in human body using recurrent neural network[J]. International Journal of Advanced Science and Technology, 2020, 29(2): 1055-1066.
[52] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[53] TOMITA N, CHEUNG Y Y, HASSANPOUR S. Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans[J]. Computers in Biology and Medicine, 2018, 98: 8-15.
[54] BAR A, WOLF L, AMITAI O B, et al. Compression fractures detection on CT[C]//Medical Imaging 2017: Computer-Aided Diagnosis, 2017: 1036-1043.
[55] SUO Q, MA F, CANINO G, et al. An attention-based recurrent neural networks framework for health data analysis[C]//Symposium on Advanced Database Systems, 2018.
[56] SUO Q, MA F, CANINO G, et al. A multi-task framework for monitoring health conditions via attention-based recurrent neural networks[C]//AMIA Annual Symposium Proceedings, 2017.
[57] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507.
[58] QADRI S F, AI D, HU G, et al. Automatic deep feature learning via patch-based deep belief network for vertebrae segmentation in CT images[J]. Applied Sciences, 2018, 9(1): 69.
[59] SHANKAR N, SATHISH BABU S, VISWANATHAN C. Femur bone volumetric estimation for osteoporosis classification using optimization-based deep belief network in X-ray images[J]. The Computer Journal, 2019, 62(11): 1656-1670.
[60] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014.
[61] LIN Y J, CHUNG I F. Medical data augmentation using generative adversarial networks: X-ray image generation for transfer learning of hip fracture detection[C]//2019 International Conference on Technologies and Applications of Artificial Intelligence, 2019: 1-5.
[62] YONG T H, YANG S, LEE S J, et al. QCBCT-NET for direct measurement of bone mineral density from quantitative cone-beam CT: a human skull phantom study[J]. Scientific Reports, 2021, 11(1): 15083.
[63] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640-651.
[64] SEKUBOYINA A, KUKA?KA J, KIRSCHKE J S, et al. Attention-driven deep learning for pathological spine segmentation[C]//International Workshop on Computational Methods and Clinical Applications in Musculoskeletal Imaging, 2017: 108-119.
[65] JIN L, YANG J, KUANG K, et al. Deep-learning-assisted detection and segmentation of rib fractures from CT scans: development and validation of FracNet[J]. EBioMedicine, 2020, 62: 103106.
[66] LESSMANN N, VAN GINNEKEN B, DE JONG P A, et al. Iterative fully convolutional neural networks for automatic vertebra segmentation and identification[J]. Medical Image Analysis, 2019, 53: 142-155.
[67] HUSSAIN D, NAQVI R A, LOH W K, et al. Deep learning in DXA image segmentation[J]. CMC Comput Mater Continua, 2021, 66(3): 2587-2598.
[68] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention, 2015: 234-241.
[69] FATHIMA S M N, TAMILSELVI R, BEHAM M P, et al. A deep learning approach on segmentation of bone for bmd measurement from dexa scan images[C]//2020 Sixth International Conference on Bio Signals, Images, and Instrumentation, 2020: 1-5.
[70] SCHMIDT D, ULéN J, ENQVIST O, et al. Deep learning takes the pain out of back breaking work-automatic vertebral segmentation and attenuation measurement for osteoporosis[J]. Clinical Imaging, 2022, 81: 54-59.
[71] ZHAO Y, ZHAO T, CHEN S, et al. Fully automated radiomic screening pipeline for osteoporosis and abnormal bone density with a deep learning-based segmentation using a short lumbar mDixon sequence[J]. Quantitative Imaging in Medicine and Surgery, 2022, 12(2): 1198.
[72] FANG Y, LI W, CHEN X, et al. Opportunistic osteoporosis screening in multi-detector CT images using deep convolutional neural networks[J]. European Radiology, 2021, 31: 1831-1842.
[73] SAEED M U, DIKAIOS N, DASTGIR A, et al. An automated deep learning approach for spine segmentation and vertebrae recognition using computed tomography images[J]. Diagnostics, 2023, 13(16): 2658.
[74] CARON R, LONDONO I, SEOUD L, et al. Segmentation of trabecular bone microdamage in Xray microCT images using a two-step deep learning method[J]. Journal of the Mechanical Behavior of Biomedical Materials, 2023, 137: 105540.
[75] YANG F, WENG X, MIAO Y, et al. Deep learning approach for automatic segmentation of ulna and radius in dual-energy X-ray imaging[J]. Insights into Imaging, 2021, 12(1): 1-9.
[76] PAN Y, SHI D, WANG H, et al. Automatic opportunistic osteoporosis screening using low-dose chest computed tomography scans obtained for lung cancer screening[J]. European Radiology, 2020, 30: 4107-4116.
[77] 黄贺瑄, 王晓燕, 顾正位, 等. 医学知识图谱构建技术及发展现状研究[J]. 计算机工程与应用, 2023, 59(13): 33-48.
HUANG H X, WANG X Y, GU Z W, et al. Research on construction technology and development status of medical knowledge graph[J]. Computer Engineering and Applications, 2023, 59(13): 33-48.
[78] 陶嘉雯, 余争平, 皮会丰. 浅析以ChatGPT为代表的人工智能对医学教育的影响[J]. 数理医药学杂志, 2023, 36(6): 475-480.
TAO J W, YU Z P, PI H F. Impact of artificial intelligence on medical education represented by ChatGPT[J]. Journal of Mathematical Medicine and Pharmacology, 2023, 36(6): 475-480.
[79] RAO A, KIM J, KAMINENI M, et al. Evaluating ChatGPT as an adjunct for radiologic decision-making[J]. medRxiv:2023.02.02.23285399.2020, 2023.
[80] JOHNSON D, GOODMAN R, PATRINELY J, et al. Assessing the accuracy and reliability of AI-generated medical responses: an evaluation of the Chat-GPT model[EB/OL]. [2023-04-06]. https://www.researchsquare.com/article/rs-2566942/v1.
[81] BENOIT J R A. ChatGPT for clinical vignette generation, revision, and evaluation[J]. medRxiv:2023.02.04.23285478, 2023.
[82] 徐璐璐, 洪贇, 叶鹰. ChatGPT及GPT类技术的医学信息学应用前景探讨[J]. 情报理论与实践, 2023, 46(6): 38-42.
XU L L, HONG Y, YE Y. Exploring the application prospect of ChatGPT and GPT type technology in medical informatics[J]. Theory and Practice of Intelligence, 2023, 46(6): 38-42.
[83] 瞿星, 杨金铭, 陈滔, 等. ChatGPT对医学教育模式改变的思考[J]. 四川大学学报 (医学版) , 2023, 54(5): 937-940.
QU X, YANG J M, CHEN T, et al. Reflections on the implications of the developments in ChatGPT for changes in medical education models [J]. Journal of Sichuan University(Medical Sciences) , 2023, 54(5): 937-940.
[84] WANG S H. OpenAI—explain why some countries are excluded from ChatGPT[J]. Nature, 2023, 615(7950): 34.
[85] BUBECK S, CHANDRASEKARAN V, ELDAN R, et al. Sparks of artificial general intelligence: early experiments with GPT-4[J]. arXiv:2303.12712, 2023.
[86] NORI H, KING N, MCKINNEY S M, et al. Capabilities of GPT-4 on medical challenge problems[J]. arXiv:2303.13375,2023.
[87] 卢经纬, 郭超, 戴星原, 等. 问答ChatGPT之后: 超大预训练模型的机遇和挑战[J]. 自动化学报, 2023, 49(4): 705-717.
LU J W, GUO C, DAI X Y, et al. The ChatGPT after: opportunities and challenges of very large scale pre-trained models[J]. Acta Automatica Sinica, 2023, 49(4): 705-717.
[88] 李克文, 于明洋. 基于双重注意力机制改进的DA-LSTM时间序列预测模型[J]. 计算机应用与软件, 2023, 40(2): 253-258.
LI K W, YU M Y. An improved DA-LSTM time series prediction model based on dual attention mechanism[J]. Computer Applications and Software, 2023, 40(2): 253-258.
[89] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[J]. arXiv:1207.0580, 2012.
[90] WAN L, ZEILER M, ZHANG S, et al. Regularization of neural networks using DropConnect[C]//International Conference on Machine Learning, 2013: 1058-1066.
[91] GOODFELLOW I, WARDE-FARLEY D, MIRZA M, et al. Maxout networks[C]//International Conference on Machine Learning, 2013: 1319-1327.
[92] 苗教伟, 季怡, 刘纯平. 基于视觉特征引导融合的视频描述方法[J]. 计算机工程与应用, 2022, 58(20): 124-131.
MIAO J W, JI Y, LIU C P. Video captioning method based on visual feature guided fusion[J]. Computer Engineering and Applications, 2022, 58(20): 124-131.
[93] 梁华刚, 雷毅雄. 增强可分离卷积通道特征的表情识别研究[J]. 计算机工程与应用, 2022, 58(2): 184-192.
LIANG H G, LEI Y X. Expression recognition with separable convolution channel enhancement features[J]. Computer Engineering and Applications, 2022, 58(2): 184-192.
[94] 微软研究院, 微软研究院公布GPT-4全面测试结果[N]. 传媒, 2023(7): 7.
MICROSOFT R. Microsoft research announces the full test results of GPT-4[N]. Media, 2023(7): 7.
[95] ZHANG J, SUN K, JAGADEESH A, et al. The potential and pitfalls of using a large language model such as ChatGPT or GPT-4 as a clinical assistant[J]. arXiv:2307.08152, 2023.