Computer Engineering and Applications ›› 2019, Vol. 55 ›› Issue (10): 146-153.DOI: 10.3778/j.issn.1002-8331.1802-0157

Previous Articles     Next Articles

Benign and Malignant Prediction of Pulmonary Lesions in Long Term Based on Multimodal Fusion

ZHANG Yanan1, ZHAO Juanjuan1, ZHAO Xin1, ZHANG Xiaolong2, WANG Sanhu3   

  1. 1.College of Computer Science and Technology, Taiyuan University of Technology, Jinzhong, Shanxi 030600, China
    2.College of Information Science and Technology, Pennsylvania State University, University Park 16802, USA
    3.Department of Computer Science and Tehnology, Luliang University, Luliang, Shanxi 033000, China
  • Online:2019-05-15 Published:2019-05-13


张娅楠1,赵涓涓1,赵  鑫1,张小龙2,王三虎3   

  1. 1.太原理工大学 计算机科学与技术学院,山西 晋中 030600
    2.宾夕法尼亚州立大学 信息科学与技术学院,尤尼弗西蒂帕克 16802
    3.吕梁学院 计算机科学与技术系,山西 吕梁 033000

Abstract: In order to characterize the changes and development rules of the lesion characteristics in various stages of pulmonary medical images more accurately and comprehensively, and study the evolution of lung nodules in the longitudinal dimension of time, this paper constructs a prediction model of benign and malignant lung lesions at different stages based on multimodal feature fusion. Firstly, according to the CT images of different stages of the patients, extract the traditional features and depth features of lung lesions, and construct the multimodal features. Then multimodal features are fused by two layers of neural networks. Finally, long and short time memory is used to study the feature vectors of lung lesions with different time characteristics. A bidirectional long and short term memory network is constructed to predict the benign and malignant lesions. The experiments show that the accuracy rate of the proposed method is 92.8%, which is higher than that of the traditional methods, and the proposed method can achieve effective prediction.

Key words: pulmonary lesions, long term, feature fusion, long and short time memory model

摘要: 为了更精确、全面地表征各时期肺部医学影像中病灶特征的变化与发展规律,研究在时间纵向维度上预测肺结节的演变方式,构建了一种多模态特征融合下不同时期肺部病灶良恶性预测模型。根据病人不同时期的序列CT图像,提取肺部病灶的传统特征与深度特征,构造多模态特征;通过神经网络对多模态特征进行相关性快速融合;利用长短时记忆方法学习不同时期具有时间特征的肺部病灶特征向量,构建一个双向长短时记忆网络对病灶进行良恶性预测。实验表明,所提方法准确率为92.8%,比传统方法有所提高,可以实现有效预测。

关键词: 肺部病灶, 长时程, 特征融合, 长短时记忆模型