基于双流关系注意力的领域自适应癌症诊断

doi:10.3778/j.issn.1002-8331.2405-0313

摘要/Abstract

摘要： 近年来，随着数字病理成像技术的飞速发展和深度学习算法的日益成熟，利用深度学习算法分析数字病理图像并对癌症进行诊断已成为研究热点。现有深度学习方法多采用全监督学习模式，其成效依赖于标记数据的数量，但病理专家精力有限，造成精准标注数据严重稀缺。为了降低标注代价，一些研究者通过设计迁移学习方法利用源域诊断信息对目标域样本标签进行预测。但现有迁移学习方法未考虑肿瘤与肿瘤浸润淋巴细胞交互作用对肿瘤诊断具有的重要参照价值。同时，由于源域和目标域数据分布存在差异，且数据质量参差不齐，直接对齐源域和目标域的特征空间可能会导致负迁移现象。针对上述问题，提出一种基于双流关系注意力的领域自适应癌症诊断方法。该方法分为图像包可转移注意力模块和示例可转移注意力模块。图像包可转移注意力模块采用双流架构，域内流利用自注意力机制学习源域和目标域内的肿瘤和肿瘤淋巴浸润细胞之间的交互关系，域间流基于交叉注意力并通过交换源域和目标域的查询集合学习域间关系。示例可转移注意力模块则用于筛选在源域和目标域上具有共性的图像块。实验结果表明该方法性能在TCGA乳腺癌和肾癌两个公共数据集的四个基准任务上优于其他领域自适应方法。

关键词: 可转移注意力, 领域自适应, 癌症诊断, 多对抗训练, 病理图像

Abstract: Recently, with the rapid development of digital photography technology and deep learning algorithms, the use of deep learning algorithms for processing and analyzing digital pathology images has become a hot topic. However, traditional fully supervised methods require a quantity of annotated data, which is labor-intensive and costly due to the diversity and heterogeneity of cancer subtypes. To reduce the cost of annotation, researchers have started exploring how to assist in the diagnosis of cancer subtypes with limited or no labeled data by transferring the knowledge learned from labeled cancer data. Nevertheless, existing transfer learning methods often overlook the similar spatial interaction relationships between tumors and tumor-infiltrating lymphocytes (TILs). In addition, the direct alignment of feature spaces between source and target domains may lead to negative transfer phenomena. Based on this, a dual-stream relation-aware attention （DRA） guided domain adaptation for cancer diagnosis is proposed. Specifically, DRA model consists of two modules to learn the transferable components: the bag-level transferable attention module (BTA) and the patch-level transferable attention module (PTA). The former BTA learns the intra-domain and inter-domain spatial interaction relationship by the self and cross-attention mechanism, respectively. Meanwhile, the latter PTA is introduced to identify the candidate patches for the spatial interaction relationship transfer. Proposed method is evaluated on two WSI datasets and the superior performance demonstrates the effectiveness of the DRA method in cross-domain cancer diagnosis.

Key words: transferable attention, domain adaption, cancer diagnosis, multi-adversarial training, histopathology images

石航, 吴雅文, 张道强, 邵伟. 基于双流关系注意力的领域自适应癌症诊断[J]. 计算机工程与应用, 2025, 61(18): 368-376.

SHI Hang, WU Yawen, ZHANG Daoqiang, SHAO Wei. Dual-Stream Relation-Aware Attention Guided Domain Adaptation for Cancer Diagnosis[J]. Computer Engineering and Applications, 2025, 61(18): 368-376.

参考文献

[1] WANG S, LIU Z Y, RONG Y, et al. Deep learning provides a new computed tomography-based prognostic biomarker for recurrence prediction in high-grade serous ovarian cancer[J]. Radiotherapy and Oncology, 2019, 132: 171-177.
[2] COUTURE H D, WILLIAMS L A, GERADTS J, et al. Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype[J]. NPJ Breast Cancer, 2018, 4: 30.
[3] COURTIOL P, MAUSSION C, MOARII M, et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome[J]. Nature Medicine, 2019, 25(10): 1519-1525.
[4] REN J, HACIHALILOGLU I, SINGER E A, et al. Unsupervised domain adaptation for classification of histopathology whole-slide images[J]. Frontiers in Bioengineering and Biotechnology, 2019, 7: 102.
[5] MAHAPATRA D, KOREVAAR S, BOZORGTABAR B, et al. Unsupervised domain adaptation using feature disentanglement and GCNs for medical image classification[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2023: 735-748.
[6] HUANG P, ZHOU X L, HE P, et al. Interpretable laryngeal tumor grading of histopathological images via depth domain adaptive network with integration gradient CAM and priori experience-guided attention[J]. Computers in Biology and Medicine, 2023, 154: 106447.
[7] WANG Z, ZHU X Y, LI A, et al. Global and local attentional feature alignment for domain adaptive nuclei detection in histopathology images[J]. Artificial Intelligence in Medicine, 2022, 132: 102341.
[8] MIYOSHI Y, SHIEN T, OGIYA A, et al. Associations in tumor infiltrating lymphocytes between clinicopathological factors and clinical outcomes in estrogen receptor-positive/human epidermal growth factor receptor type 2 negative breast cancer[J]. Oncology Letters, 2019, 17(2): 2177-2186.
[9] YOSINSKI J, CLUNE J, BENGIO Y, et al. How transferable are features in deep neural networks? [J]. arXiv:1411.1792, 2014.
[10] YANG J Y, LIU J J, XU N, et al. TVT: transferable vision transformer for unsupervised domain adaptation[C]//Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2023: 520-530.
[11] XU T K, CHEN W H, WANG P C, et al. CDTrans: cross-domain transformer for unsupervised domain adaptation[J]. arXiv:2109.06165, 2021.
[12] DALLE J R, LEOW W K, RACOCEANU D, et al. Automatic breast cancer grading of histopathological images[C]//Proceedings of the 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway: IEEE, 2008: 3052-3055.
[13] WAN T, CAO J J, CHEN J H, et al. Automated grading of breast cancer histopathology using cascaded ensemble with combination of multi-level image features[J]. Neurocomputing, 2017, 229: 34-44.
[14] ERTOSUN M G, RUBIN D L. Automated grading of gliomas using deep learning in digital pathology images: a modular approach with ensemble of convolutional neural networks[C]//AMIA Annual Symposium Proceedings, 2015: 1899-1908.
[15] LU C, ROMO-BUCHELI D, WANG X X, et al. Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers[J]. Laboratory Investigation, 2018, 98(11): 1438-1448.
[16] PERINCHERI S, LEVI A W, CELLI R, et al. An independent assessment of an artificial intelligence system for prostate cancer detection shows strong diagnostic accuracy[J]. Modern Pathology, 2021, 34(8): 1588-1595.
[17] PANTANOWITZ L, QUIROGA-GARZA G M, BIEN L, et al. An artificial intelligence algorithm for prostate cancer diagnosis in whole slide images of core needle biopsies: a blinded clinical validation and deployment study[J]. The Lancet Digital Health, 2020, 2(8): e407-e416.
[18] STR?M P, KARTASALO K, OLSSON H, et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study[J]. The Lancet Oncology, 2020, 21(2): 222-232.
[19] PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[20] LIU B. Lifelong machine learning: a paradigm for continuous learning[J]. Frontiers of Computer Science, 2017, 11(3): 359-361.
[21] HUANG Y, ZHENG H, LIU C, et al. Epithelium-stroma classification via convolutional neural networks and unsupervised domain adaptation in histopathological images[J]. IEEE Journal of Biomedical and Health Informatics, 2017, 21(6): 1625-1632.
[22] YU X, ZHENG H, LIU C, et al. Classify epithelium-stroma in histopathological images based on deep transferable network[J]. Journal of Microscopy, 2018, 271(2): 164-173.
[23] TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial discriminative domain adaptation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2962-2971.
[24] CHEN C Q, XIE W P, WEN Y, et al. Multiple-source domain adaptation with generative adversarial nets[J]. Knowledge-Based Systems, 2020, 199: 105962.
[25] GANIN Y, LEMPITSKY V S. Unsupervised domain adaptation by backpropagation[C]//Proceedings of the International Conference on Machine Learning, 2014.
[26] ZUO Y L, WU Y W, LU Z X, et al. Identify consistent imaging genomic biomarkers for characterizing the survival-associated interactions between tumor-infiltrating lymphocytes and tumors[C]//Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention. Cham: Springer, 2022: 222-231.
[27] BERMAN M, TRIKI A R, BLASCHKO M B. The lovasz-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4413-4421.
[28] ZHANG A R, XU J, LUO X Y, et al. Cross-domain attention network for unsupervised domain adaptation crowd counting[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(10): 6686-6699.
[29] LONG M S, WANG J M, DING G G, et al. Transfer joint matching for unsupervised domain adaptation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2014: 1410-1417.
[30] GONG B Q, SHI Y, SHA F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2012: 2066-2073.
[31] PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis[J]. IEEE Transactions on Neural Networks, 2011, 22(2): 199-210.
[32] TZENG E, HOFFMAN J, ZHANG N, et al. Deep domain confusion: maximizing for domain invariance[J]. arXiv:1412.3474, 2014.
[33] GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research, 2016, 17(59): 1-35.
[34] SHEN J, QU Y R, ZHANG W N, et al. Wasserstein distance guided representation learning for domain adaptation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2018.
[35] DAMODARAN B B, KELLENBERGER B, FLAMARY R, et al. DeepJDOT: deep joint distribution optimal transport for unsupervised domain adaptation[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2018: 467-483.
[36] ZHANG Y C, LIU T L, LONG M S, et al. Bridging theory and algorithm for domain adaptation[C]//Proceedings of the International Conference on Machine Learning, 2019.