CNN-Transformer交互模型预测IgA肾病病理分级

doi:10.3778/j.issn.1002-8331.2402-0041

摘要/Abstract

摘要： 肾脏超声影像检查是IgA肾病重要的临床无创诊断方法，可以避免不必要的肾活检，尤其对患者长期病程管理至关重要。但是超声影像解析与肾活检病理分析仍具有巨大知识鸿沟，导致临床上难以直接通过超声影像解析进行精准的IgA肾病病理分级，其诊断仍然高度依赖于IgA肾病病理的分析。融合卷积神经网络（convolutional neural network，CNN）和Transformer设计了一种CNN-Transformer交互模型，通过肾脏超声影像的自动解析，实现IgA肾病病理分级预测，辅助医生对疾病进行诊断。在该交互模型中，为了模拟医生阅片观察肾脏皮髓质区的局部纹理形态，融入可变形卷积设计了CNN结构流提取肾脏局部特征；为了模拟医生阅片观察肾脏整体组织形态，融入空间偏置注意力设计了Transformer结构流提取肾脏组织全局关联关系；为了模拟医生阅片反复交替观察各组织形态进行综合评估的过程，提出了一种中期特征交互策略和一种终端自适应融合策略，全面揭示超声影像蕴含的医学信息。实验结果表明，所提出的CNN-Transformer交互模型能够有效地通过肾脏超声影像进行IgA肾病病理分级，准确率达到0.842，灵敏度达到0.833，特异性达到0.876，AUC达到0.931，优于多个现有超声影像深度学习方法。

关键词: IgA肾病, 超声, 深度学习, 混合模型, 双向交互, 自适应融合

Abstract: Renal ultrasound examination is an important non-invasive clinical diagnostic method for IgA nephropathy (IgAN), which can avoid unnecessary renal biopsy and is especially crucial for long-term disease management. However, there is still a huge knowledge gap between ultrasound image analysis and renal biopsy pathology analysis, which makes it difficult to perform accurate IgAN pathology stratification directly from ultrasound images in clinical practice. Thus, IgAN diagnosis is still highly dependent on the analysis of its pathology. In this paper, a CNN-Transformer interaction model is designed by fusing convolutional neural network (CNN) and Transformer to achieve the prediction of IgAN pathology stratification through the automatic parsing of renal ultrasound images, which assists doctors in its diagnosis. In this model, to simulate the doctor observing the local texture morphology of the corticomedullary region of the kidney, a CNN stream is designed by integrating deformable convolution to extract the local features of the kidney. To simulate the observations of the overall tissue morphology of the kidney, a Transformer stream is designed with spatial biased attention to extract the global correlations of the renal tissues. To simulate the alternate observations of renal tissue morphology for comprehensive assessment, an intermediate-term feature interaction strategy and a terminal adaptive fusion strategy are proposed to comprehensively reveal the medical information contained in ultrasound images. The experimental results show that the proposed CNN-Transformer interaction model performs well in IgAN pathology stratification through renal ultrasound images with an accuracy of 0.842, a sensitivity of 0.833, a specificity of 0.876, and an AUC of 0.931, which outperforms several existing deep learning methods for ultrasound images.

Key words: IgA nephropathy, ultrasound, deep learning, hybrid model, bidirectional interaction, adaptive fusion

牛昊天, 林宇轩, 蔡念, 谢依颖, 张镭. CNN-Transformer交互模型预测IgA肾病病理分级[J]. 计算机工程与应用, 2025, 61(10): 331-340.

NIU Haotian, LIN Yuxuan, CAI Nian, XIE Yiying, ZHANG Lei. CNN-Transformer Interaction Model Prediction Pathological Stratification of IgA Nephropathy[J]. Computer Engineering and Applications, 2025, 61(10): 331-340.

参考文献

[1] PATTRAPORNPISUT P, AVILA-CASADO C, REICH H N. IgA nephropathy: core curriculum 2021[J]. American Journal of Kidney Diseases, 2021, 78(3): 429-441.
[2] WU H Y, XIA Z K, GAO C L, et al. The correlation analysis between the Oxford classification of Chinese IgA nephropathy children and renal outcome - a retrospective cohort study[J]. BMC Nephrology, 2020, 21(1): 247.
[3] AVASARE R S, ROSENSTIEL P E, ZAKY Z S, et al. Predicting post-transplant recurrence of IgA nephropathy: the importance of crescents[J]. American Journal of Nephrology, 2017, 45(2): 99-106.
[4] CATTRAN D C, COPPO R. The Oxford classification of IgA nephropathy: rationale, clinicopathological correlations, and classification[J]. Kidney International, 2009, 76(5): 534-545.
[5] TRIMARCHI H, BARRATT J, CATTRAN D C, et al. Oxford classification of IgA nephropathy 2016: an update from the IgA nephropathy classification working group[J]. Kidney International, 2017, 91(5): 1014-1021.
[6] YOON C Y, CHANG T I, KANG E W, et al. Clinical usefulness of the Oxford classification in determining immunosuppressive treatment in IgA nephropathy[J]. Annals of Medicine, 2017, 49(3): 217-229.
[7] Kidney Disease: Improving Global Outcomes (KDIGO) Glomerular Diseases Work Group. KDIGO 2021 clinical practice guideline for the management of glomerular diseases[J]. Kidney International, 2021, 100(4S): S1-S276.
[8] HAASKJOLD Y L, BJ?RNEKLETT R, BOSTAD L, et al. Utilizing the MEST score for prognostic staging in IgA nephropathy[J]. BMC Nephrology, 2022, 23(1): 26.
[9] PURWAR S, TRIPATHI R, BARWAD A W, et al. Detection of Mesangial hypercellularity of MEST-C score in immunoglobulin A-nephropathy using deep convolutional neural network[J]. Multimedia Tools and Applications, 2020, 79(37): 27683-27703.
[10] WEIS C A, BINDZUS J N, VOIGT J, et al. Assessment of glomerular morphological patterns by deep learning algorithms[J]. Journal of Nephrology, 2022, 35(2): 417-427.
[11] ALTINI N, ROSSINI M, TURKEVI-NAGY S, et al. Performance and limitations of a supervised deep learning approach for the histopathological Oxford classification of glomeruli with IgA nephropathy[J]. Computer Methods and Programs in Biomedicine, 2023, 242: 107814.
[12] LUCIANO R L, MOECKEL G W. Update on the native kidney biopsy: core curriculum 2019[J]. American Journal of Kidney Diseases, 2019, 73(3): 404-415.
[13] MAVROGEORGIS E, HE T L, MISCHAK H, et al. Urinary peptidomic liquid biopsy for non-invasive differential diagnosis of chronic kidney disease[J]. Nephrology, Dialysis, Transplantation, 2024, 39(3): 453-462.
[14] DHINDSA K, SMAIL L C, MCGRATH M, et al. Grading prenatal hydronephrosis from ultrasound imaging using deep convolutional neural networks[C]//Proceedings of the 2018 15th Conference on Computer and Robot Vision. Piscataway: IEEE, 2018: 80-87.
[15] KUO C C, CHANG C M, LIU K T, et al. Automation of the kidney function prediction and classification through ultrasound-based kidney imaging using deep learning[J]. NPJ Digital Medicine, 2019, 2: 29.
[16] NADEEM M, TAN G Z, ALTAF M, et al. Evaluation and classification of kidney stone detection using deep learning techniques[C]//Proceedings of the 2023 6th International Conference on Software Engineering and Computer Science. Piscataway: IEEE, 2023: 1-6.
[17] MANGAYARKARASI T, NAJUMNISSA JAMAL D. Kidney abnormalities prediction in ultrasound images using transfer learning approach[C]//Proceedings of the 2023 2nd International Conference on Advances in Computational Intelligence and Communication. Piscataway: IEEE, 2023: 1-6.
[18] 李广涵, 刘健, 马立勇, 等. 基于深度学习的超声影像诊断对终末期慢性肾病的价值[J]. 中华医学超声杂志 (电子版), 2021, 18(6): 611-615.
LI G H, LIU J, MA L Y, et al. Deep learning-based model for ultrasound diagnosis of end-stage chronic kidney disease[J]. Chinese Journal of Medical Ultrasound (Electronic Edition), 2021, 18(6): 611-615.
[19] TANG Z Y, LIN Y C, SHEN C C. Dual-path convolutional neural network for chronic kidney disease classification in ultrasound echography[C]//Proceedings of the 2022 IEEE International Ultrasonics Symposium. Piscataway: IEEE, 2022: 1-4.
[20] GHEFLATI B, RIVAZ H. Vision transformers for classification of breast ultrasound images[C]//Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society. Piscataway: IEEE, 2022: 480-483.
[21] HUANG H, DONG Y J, JIA X H, et al. Personalized diagnostic tool for thyroid cancer classification using multi-view ultrasound[C]//Proceedings of the 25th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2022: 665-674.
[22] SARKER M M K, SINGH V K, ALSHARID M, et al. COMFormer: classification of maternal-fetal and brain anatomy using a residual cross-covariance attention guided transformer in ultrasound[J]. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2023, 70(11): 1417-1427.
[23] DAI J F, QI H Z, XIONG Y W, et al. Deformable convolutional networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 764-773.
[24] KINGMA D P, BA J, HAMMAD M M. Adam: a method for stochastic optimization[J]. arXiv:1412.6980, 2014.
[25] PENG Z L, HUANG W, GU S Z, et al. Conformer: local features coupling global representations for visual recognition[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 357-366.
[26] JIANG J Y, ZHANG P Y, LUO Y T, et al. AdaMCT: adaptive mixture of CNN-transformer for sequential recommendation[C]//Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. New York: ACM, 2023: 976-986.
[27] SHAREEF B, XIAN M, VAKANSKI A, et al. Breast ultrasound tumor classification using a hybrid multitask CNN-transformer network[C]//Proceedings of the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2023: 344-353.