Abstract: In order to characterize the changes and development rules of the lesion characteristics in various stages of pulmonary medical images more accurately and comprehensively, and study the evolution of lung nodules in the longitudinal dimension of time, this paper constructs a prediction model of benign and malignant lung lesions at different stages based on multimodal feature fusion. Firstly, according to the CT images of different stages of the patients, extract the traditional features and depth features of lung lesions, and construct the multimodal features. Then multimodal features are fused by two layers of neural networks. Finally, long and short time memory is used to study the feature vectors of lung lesions with different time characteristics. A bidirectional long and short term memory network is constructed to predict the benign and malignant lesions. The experiments show that the accuracy rate of the proposed method is 92.8%, which is higher than that of the traditional methods, and the proposed method can achieve effective prediction.

Key words: pulmonary lesions, long term, feature fusion, long and short time memory model

摘要： 为了更精确、全面地表征各时期肺部医学影像中病灶特征的变化与发展规律，研究在时间纵向维度上预测肺结节的演变方式，构建了一种多模态特征融合下不同时期肺部病灶良恶性预测模型。根据病人不同时期的序列CT图像，提取肺部病灶的传统特征与深度特征，构造多模态特征；通过神经网络对多模态特征进行相关性快速融合；利用长短时记忆方法学习不同时期具有时间特征的肺部病灶特征向量，构建一个双向长短时记忆网络对病灶进行良恶性预测。实验表明，所提方法准确率为92.8%，比传统方法有所提高，可以实现有效预测。

关键词: 肺部病灶, 长时程, 特征融合, 长短时记忆模型