关键词: 膝关节损伤, 磁共振成像（MRI）影像, 多模态, 特征融合

Abstract:

Magnetic Resonance Imaging（MRI） of the knee is the preferred method for diagnosing knee injuries. However, the interpretation of knee MRI is time-intensive and prone to errors. In order to assist clinicians to diagnose more accurately, a deep learning model based on multimodal fusion is proposed to detect general anomalies, Anterior Cruciate Ligament（ACL） tears and meniscus tears. HOG features and LBP features are contacted as low-level semantic features after selecting the features with over 95% contribution by PCA. Feature pyramid network on the basis of VGG16 model merges multiple feature maps as high-level semantic features. The low-level semantic features and the high-level semantic features are fused by correlation to get the prediction probability through an energy model which consists of multi-layer neural network. The experiments show that the AUC of the general anomaly, ACL tear and meniscus tear are 0.941?0, 0.970?8 and 0.8479, which have certain advantages over the results of the high-level semantic features or the low-level semantic features.

Key words: knee injury, Magnetic Resonance Imaging（MRI） image, multimodality, feature fusion