基于深度学习的医学实体和关系联合抽取研究综述

doi:10.3778/j.issn.1002-8331.2403-0457

摘要/Abstract

摘要： 命名实体识别与关系抽取作为医学领域信息抽取的核心任务，能够从非结构化或半结构化的文本中自动识别实体边界、实体类型以及实体之间的关系。不仅能够促进知识的发现与整合，应用于临床决策，加强药物的发现和再利用，还可以助力公共卫生监测和疾病预防。回顾了实体识别和关系抽取的发展历程，介绍了常用评价指标和医学领域实体关系联合抽取数据集，指出目前联合抽取领域存在医学文本结构比较复杂、实体关系重叠句子抽取率低等问题。根据这些问题，进一步探讨了基于深度学习的实体关系联合抽取方法在医学领域上的应用。这些方法根据模型解码的方式主要分为基于共享参数的联合抽取模型和基于联合解码的联合抽取模型，从问题解决角度对不同的模型的优缺点进行探讨分析和总结。讨论了医学领域实体关系抽取面临的挑战和未来的研究方向。

关键词: 医学文本, 联合抽取, 关系抽取, 实体识别

Abstract: Named entity recognition and relationship extraction are core tasks in the field of medical information extraction. They enable the automatic identification of entities, entity types, and relationships between entities from unstructured or semi-structured text. This capability not only facilitates the discovery and integration of knowledge, application in clinical decision-making, and enhancement of drug discovery and repurposing, but also supports public health monitoring and disease prevention. This article begins by reviewing the development of entity recognition and relationship extraction, introducing common evaluation metrics and datasets for joint entity and relationship extraction in the medical field. It highlights current challenges in the field, such as the complexity of medical text structures and the low accuracy of joint extraction. Building on these issues, the article further explores the application of deep learning-based methods for joint entity and relationship extraction in the medical field. These methods are primarily categorized into joint extraction models based on shared parameters and those based on joint decoding. The article discusses and summarizes the advantages and disadvantages of different models from a problem-solving perspective. Finally, the article discusses the challenges in joint entity and relationship extraction within the medical field and suggests future research directions.

Key words: medical text, joint extraction of entity and relation, relation extraction, named entity recognition

叶青, 张晓凤, 彭琳, 程春雷. 基于深度学习的医学实体和关系联合抽取研究综述[J]. 计算机工程与应用, 2024, 60(24): 65-78.

YE Qing, ZHANG Xiaofeng, PENG Lin, CHENG Chunlei. Review of Joint Extraction of Medical Entities and Relationships Based on Deep Learning[J]. Computer Engineering and Applications, 2024, 60(24): 65-78.

参考文献

[1] 鄂海红, 张文静, 肖思琪, 等. 深度学习实体关系抽取研究综述[J]. 软件学报, 2019, 30(6): 1793-1818.
E H H, ZHANG W J, XIAO S Q, et al. Survey of entity relationship extraction based on deep learning[J]. Journal of Software, 2019, 30(6): 1793-1818.
[2] 刘辉, 江千军, 桂前进, 等. 实体关系抽取技术研究进展综述[J]. 计算机应用研究, 2020, 37(S2): 1-5.
LIU H, JIANG Q J, GUI Q J, et al. Review of research progress of entity relationship extraction[J]. Application Research of Computers, 2020, 37(S2): 1-5.
[3] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251.
ZHOU F Y, JIN L P, DONG J. Review of convolutional neural network[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251.
[4] 隗昊, 周爱, 张益嘉, 等. 深度学习生物医学实体关系抽取研究综述[J]. 计算机工程与应用, 2021, 57(21): 14-23.
WEI H, ZHOU A, ZHANG Y J, et al. Review of deep learning-based biomedical entity relation extraction research[J]. Computer Engineering and Applications, 2021, 57(21): 14-23.
[5] 易钧汇, 查青林. 中医症状信息抽取研究综述[J]. 计算机工程与应用, 2023, 59(17): 35-47.
YI J H, Z Q L. Survey of TCM symptom information extraction[J]. Computer Engineering and Applications, 2023, 59(17): 35-47.
[6] 张少伟, 王鑫, 陈子睿, 等. 有监督实体关系联合抽取方法研究综述[J]. 计算机科学与探索, 2022, 16(4): 713-733.
ZHANG S W, WANG X, CHEN Z R, et al. Survey of supervised joint entity relation extraction methods[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 713-733.
[7] KHALIFA M, SHAALAN K. Character convolutions for Arabic named entity recognition with long short-term memory networks[J]. Computer Speech and Language, 2019, 58(3): 335-346.
[8] JIANG, M, CHEN Y, LIU M, et al. A study of machine-learning-based approaches to extract clinical entities and their assertions from discharge summaries[J]. Journal of the American Medical Informatics Association, 2011, 18: 601-606.
[9] LEI J B, TANG B Z, LIU XQ, et al. A comprehensive study of named entity recognition in Chinese clinical text[J]. Journal of the American Medical Informatics Association, 2014, 21(5): 808-814.
[10] UGUZ H, OZTURK A, SARA?OGLU R, et al. A biomedical system based on fuzzy discrete hidden Markov model for the diagnosis of the brain diseases[J]. Expert Systems with Applications, 2007, 35(3): 1104-1114.
[11] SHERSTINSKY A. Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network[J]. Physica D Nonlinear Phenomena, 2020, 404(8): 132306.
[12] DEVLIN J, CHANG M W, LEE K, et al. Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Minnesota, Jun, 2019. Stroudsburg: ACL, 2019: 4171-4186.
[13] LIU Z, YANG M, WANG X, et al. Entity recognition from clinical texts via recurrent neural network[J]. BMC Medical Informatics and Decision Making, 2017, 17: 53-61.
[14] GRIDACH M. Character-level neural network for biomedical named entity recognition[J]. Journal of Biomedical Informatics, 2017, 70: 85-91.
[15] ZHANG Y, YANG J. Chinese NER using lattice LSTM[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, Jul, 2018. Stroudsburg: ACL, 2018: 1554-1564.
[16] ZHAO S, CAI Z, CHEN H, et al. Adversarial training based lattice LSTM for Chinese clinical named entity recognition[J]. Journal of Biomedical Informatics, 2019, 99: 103290.
[17] LI X, ZHANG H, ZHOU X H. Chinese clinical named entity recognition with variant neural structures based on BERT methods[J]. Journal of Biomedical Informatics, 2020, 107: 103422.
[18] GAO W C, ZHENG X H, ZHAO S S. Named entity recognition method of Chinese EMR based on BERT-BiLSTM-CRF[J]. Journal of Physics: Conference Series, 2021, 1848(1): 012083.
[19] WANG J, XU W, FU X, et al. ASTRAL: adversarial trained LSTM-CNN for named entity recognition[J]. Knowledge-Based Systems, 2020, 197: 105842.
[20] LI F, ZHANG M, FU G, et al. A neural joint model for entity and relation extraction from biomedical text[J]. BMC Bioinformatics, 2017, 18: 1-11.
[21] ZHANG Y, LIN H, YANG Z, et al. A hybrid model based on neural networks for biomedical relation extraction[J]. Journal of Biomedical Informatics, 2018, 81: 83-92.
[22] TIAN Y, CHEN G, SONG Y, et al. Dependency-driven relation extraction with attentive graph convolutional networks[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg: ACL, 2021: 4458-4471.
[23] WANG D, TIWARI P, GARG S, et al. Structural block driven enhanced convolutional neural representation for relation extraction[J]. Applied Soft Computing, 2020, 86: 105913.
[24] ZHANG X, ZHANG Y, ZHANG Q, et al. Extracting comprehensive clinical information for breast cancer using deep learning methods[J]. International Journal of Medical Informatics, 2019, 132: 103985.
[25] XU S, SUN S, ZHANG Z, et al. BERT gated multi-window attention network for relation extraction[J]. Neurocomputing, 2022, 492: 516-529.
[26] LEE J, YOON W, KIM S, et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining[J]. Bioinformatics, 2020, 36(4): 1234-1240.
[27] GUAN T, ZAN H, ZHOU X, et al. CMeIE: construction and evaluation of Chinese medical information extraction dataset[C]//Proceedings of the 9th CCF International Conference on Natural Language Processing and Chinese Computing, Zhengzhou, China, Oct 14-18, 2020. Berlin: Springer, 2020: 270-282.
[28] HERRERO-ZAZO M, SEGURA-BEDMAR I, MARTíNEZ P, et al. The DDI corpus: an annotated corpus with pharmacological substances and drug-drug interactions[J]. Journal of Biomedical Informatics, 2013, 46(5): 914-920.
[29] GURULINGAPPA H, RAJPUT A M, ROBERTS A, et al. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports[J]. Journal of Biomedical Informatics, 2012, 45(5): 885-892.
[30] LI J, SUN Y, JOHNSON R J, et al. BioCreative V CDR task corpus: a resource for chemical disease relation extraction[J]. Database: the Journal of Biological Databases and Curation, 2016: 068.
[31] PYYSALO S, GINTER F, HEIMONEN J, et al. Bioinfer: a corpus for information extraction in the biomedical domain[J]. BMC Bioinforma, 2007, 8: 266-267.
[32] SUN W, RUMSHISKY A, UZUNER O. Evaluating temporal relations in clinical text: 2012 i2b2 challenge[J]. Journal of the Ameriean Medical Informaties Association, 2013, 20(5): 806-813.
[33] MAKOTO M, MOHIT B. End-to-end relation extraction using LSTMs on sequences and tree structures[J]. Computing Research Repository, 2016, 1: 1105-1116.
[34] ZHENG S C, XU J M, BAO H Y, et al. Joint learning of entity semantics and relation pattern for relation extraction[C]//Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 2016: 443-458.
[35] GIANNIS B, JOHANNES D, THOMAS D, et al. Joint entity recognition and relation extraction as a multi-head selection problem[J]. Computing Research Repository, 2018, 114: 34-45.
[36] SUN Q, ZHANG K, LV L S, et al. Joint extraction of entities and overlapping relations by improved graph convolutional networks[J]. Applied Intelligence, 2021, 52(5): 5212-5224.
[37] 李代祎, 李忠良, 严丽. 一种面向中文的实体关系联合抽取方法研究[J]. 小型微型计算机系统, 2022, 43(12): 2479-2486.
LI D Y, LI Z L, YAN L. Research on Chinese-oriented entity relation joint extraction method[J]. Journal of Chinese Computer Systems, 2022, 43(12): 2479-2486.
[38] 董美, 常志军. 一种面向中医领域科技文献的实体关系抽取方法[J]. 图书情报工作, 2022, 66(18): 105-113.
DONG M, CHANG Z J. An entity relation extraction method for scientific and technological documents in the field of traditional Chinese medicinel[J]. Library and Information Service, 2022, 66 (18): 105-113.
[39] 李丽双, 王泽昊, 秦雪洋, 等. 基于平行交互注意力网络的中文电子病历实体及关系联合抽取[J]. 中文信息学报, 2024, 38(6): 108-118.
LI L S, WANG Z H, QIN X Y. Parallel interactive attention network for joint entity and relation extraction based on Chinese electronic medical record[J]. Journal of Chinese Information Processing, 2024, 38(6): 108-118.
[40] LI X Y, YIN F, SUN Z J, et al. Entity-relation extraction as multi-turn question answering[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, Jul, 2019. Stroudsburg: ACL, 2019: 1340-1350.
[41] ZHAO T Y, YAN Z, CAO Y B, et al. A unified multi-task learning framework for joint extraction of entities and relations[C]//Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, Canada, Feb 2-9, 2021. Menlo Park: AAAI, 2021: 14524-14531.
[42] 曹明宇, 杨志豪, 罗凌, 等. 基于神经网络的药物实体与关系联合抽取[J]. 计算机研究与发展, 2019, 56(7): 1432-1440.
CAO M Y, YANG Z H, LUO L, et al. Joint drug entities and relations extraction based on neural networks[J]. Journal of Computer Research and Development, 2019, 56(7): 1432-1440.
[43] ZHENG S, WANG F, BAO H, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, Jul, 2017. Stroudsburg: ACL, 2017: 1227-1236.
[44] LUO L, YANG Z H, CAO M Y, et al. A neural network-based joint learning approach for biomedical entity and relation extraction from biomedical literature[J]. Journal of Biomedical Informatics, 2020, 103(C): 103384.
[45] TAO Z H, OUYANG C P, LIU Y B, et al. Multi-head attention graph convolutional network model: end-to-end entity and relation joint extraction based on multi-head attention graph convolutional network[J]. CAAI Transactions on Intelligence Technology, 2022, 8(2): 468-477.
[46] SUN C, YANG Z H, WANG L, et al. MRC4BioER: joint extraction of biomedical entities and relations in the machine reading comprehension framework[J]. Journal of Biomedical Informatics, 2022, 125: 103956.
[47] 杨延云, 杜建强, 聂斌, 等. 一种面向中医文本的实体关系深度学习联合抽取方法[J]. 计算机应用与软件, 2023, 40(3): 217-222.
YANG Y Y, DU J Q, NIE B, et al. A joint extraction method of entities and relations for traditional Chinese medicine text based on deep learning[J]. Computer Applications and Software, 2023, 40(3): 217-222.
[48] 刘珂, 冷芳玲, 施伯贻, 等. 基于深度主动学习的实体关系联合抽取方法[J]. 郑州大学学报 (理学版), 2023, 55(3): 34-40.
LIU K, LENG F L, SHI B Y, et al. Joint extraction method of entities and relations based on deep active learning[J]. Journal of Zhengzhou University (Natural Science Edition), 2023, 55(3): 34-40.
[49] DAI D, XIAO X, LYU Y, et al. Joint extraction of entities and overlapping relations using position-attentive sequence labeling[C]//Proceedings of the AAAI Conference on Artificial Intelligence, Hawaii, USA, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 6300-6308.
[50] WEI Z, SU J, WANG Y, et al. A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul, 2020. Stroudsburg: ACL, 2020: 1476-1488.
[51] SHI B, FAN R, ZHANG L, et al. A joint extraction system based on conditional layer normalization for health monitoring[J]. Sensors, 2023, 23(10): 4812.
[52] 仲雨乐, 韩普, 许鑫. 基于异构图注意力网络的药物不良反应实体关系联合抽取研究[J]. 现代情报, 2024(9): 71-81.
ZHONG Y L, HAN P, XU X. Joint extraction of adverse drug reactions entities and relations based on heterogeneous graph attention network[J]. Journal of Modern Information, 2024(9): 71-81.
[53] 王春亮, 姚洁仪, 李昭. 融合MacBERT和Talking-Heads Attention实体关系联合抽取模型[J]. 现代电子技术, 2024, 47(5): 127-131.
WANG C L, YAO J Y, LI Z. Entity relation joint extraction model fusing MacBERT and talking-heads attention[J]. Modern Electronics Technique, 2024, 47(5): 127-131. `
[54] 余肖生, 李琳宇, 周佳伦, 等. AMFRel: 一种中文电子病历实体关系联合抽取方法[J]. 重庆理工大学学报 (自然科学), 2024, 38(2): 189-197.
YU X S, LI L Y, ZHOU J L, et al. AMFRel: a method for joint extraction of entity relations in Chinese electronic medical records[J]. Journal of Chongqing University of Technology (Natural Science), 2024, 38(2): 189-197.
[55] BAI T, GUAN H T, WANG S, et al. Traditional Chinese medicine entity relation extraction based on CNN with segment attention[J]. Neural Computing and Applications, 2021, 34(4): 2739-2748.
[56] 宁尚明, 滕飞, 李天瑞. 基于多通道自注意力机制的电子病历实体关系抽取[J]. 计算机学报, 2020, 43(5): 916-929.
NING S M, TENG F, LI T R. Multi-channel self-attention mechanism for relation extraction in clinical records[J]. Chinese Journal of Computers, 2020, 43(5): 916-929.
[57] HE K, MAO R, GONG T L, et al. JCBIE: a joint continual learning neural network for biomedical information extraction[J]. BMC Bioinformatics, 2022, 23(1): 549.
[58] LI J, XU Y, LIN H, et al. Semantic-consistent learning for one-shot joint entity and relation extraction[J]. Applied Intelligence, 2023, 53(5): 5963-5976.
[59] ZENG X, ZENG D, HE S, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, Jul, 2018. Stroudsburg: ACL, 2018: 506-514.
[60] JOHN G, GARY D B, BO W. A sequence-to-sequence approach for document-level relation extraction[C]//Proceedings of the 21st Workshop on Biomedical Language Processing, Dublin, Ireland, May 2022. Stroudsburg: ACL, 2022: 10-25.
[61] LIN Y K, SHEN S Q, LIU Z Y, et al. Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, Aug, 2016. Stroudsburg: ACL, 2016: 2124-2133.
[62] SHANG Y M, HUANG H Y, SUN X, et al. A pattern-aware self-attention network for distant supervised relation extraction[J]. Information Sciences, 2022, 584: 269-279.
[63] LIU R R, MO S S, NIU J W, et al. CETA: a consensus enhanced training ap proach for denoising in distantly supervised rela tion extraction[C]//Proceedings of the 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea, Oct, 2022. Stroudsburg: ACL, 2022: 2247-2258.
[64] LI Y, YU X, LIU Y, et al. Uncertainty-aware bootstrap learning for joint extraction on distantly-supervised data[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, Canada, Jul, 2023. Stroudsburg: ACL, 2023: 1349-1358.
[65] LU D, NEVES L, CARVALHO V, et al. Visual attention model for name tagging in multimodal social media[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia, Jul, 2018. Stroudsburg: ACL, 2018: 1990-1999.
[66] MOON S, NEVES L, CARVALHO V. Multimodal named entity recognition for short social media posts[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, Jun 2018. Stroudsburg: ACL, 2018: 852-860.
[67] WU Z, ZHENG C, CAI Y, et al. Multimodal representation with embedded visual guiding objects for named entity recognition in social media posts[C]//Proceedings of the 28th ACM International Conference on Multimedia, Seattle, USA, Oct 12-16, 2020. New York: ACM, 2020: 1038-1046.
[68] YU J, JIANG J, YANG L, et al. Improving multimodal named entity recognition via entity span detection with unified multimodal transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul, 2020. Stroudsburg: ACL, 2020: 3342-3352.
[69] YUAN L, CAI Y, WANG J, et al. Joint multimodal entity-relation extraction based on edge-enhanced graph alignment network and word-pair relation tagging[C]//Proceedings of the AAAI Conference on Artificial Intelligence, Washington, USA, Feb 7-14, 2023. Menlo Park: AAAI, 2023: 11051-11059.
[70] ZHENG C, FENG J, FU Z, et al. Multimodal relation extraction with efficient graph alignment[C]//Proceedings of the 29th ACM International Conference on Multimedia, Oct 20-24, 2021. New York: Association for Computing Machinery, 2021: 5298-5306.
[71] ZHENG C, WU Z, FENG J, et al. MNRE: a challenge multimodal dataset for neural relation extraction with visual evidence in social media posts[C]//Proceedings of the 2021 IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China, 2016. Piscataway: IEEE, 2021: 1-6.
[72] AINSLIE J, LEE-THORP J, DE JONG M, et al. GQA: training generalized multi-query transformer models from multi-head checkpoints[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, Dec 6-10, 2023. Stroudsburg: ACL, 2023: 4895-4901.
[73] SINGHAL K, AZIZI S, TU T, et al. Large language models encode clinical knowledge[J]. Nature, 2023, 620(7972): 172-180.
[74] HE M, BAI Y. LAL-JER: label-aware learning for adaptive joint entity and relation extraction with LLM data augmentation[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, Dec 6-10, 2023. Stroudsburg: ACL, 2023: 414-419.
[75] FRANTAR E, ALISTARH D. SparseGPT: massive language models can be accurately pruned in one-shot[C]//Proceedings of the International Conference on Machine Learning, Hawaii, USA, Jul 23-29, 2023. New York: PMLR, 2023: 10323-10337.
[76] MA X, FANG G, WANG X. LLM-pruner: on the structural pruning of large language models[C]//Advances in Neural Information Processing Systems, 2023: 21702-21720.