Medical Named Entity Recognition Incorporating Word Information and Graph Attention

doi:10.3778/j.issn.1002-8331.2302-0321

Abstract

Abstract: The Chinese clinical natural language is rich in a large amount of medical record information. Naming entity recognition for electronic medical records can help establish medical auxiliary diagnostic systems, which is of great significance for the development of the medical field. At the same time, it is conducive to downstream tasks such as relationship extraction and the implementation of knowledge graphs. However, Chinese electronic medical records have problems with difficulty in Chinese word segmentation, numerous medical terminology, and special expressions, which can easily lead to incorrect expression of text features. Therefore, this paper proposes a medical named entity recognition research model based on enhanced word information and graph attention, which improves the performance of the network model by enhancing local and global features. Due to the fact that embedding a single word vector for Chinese entity recognition can easily ignore word information and semantics in the text, this paper embeds a highly correlated word vector in the word vector, which not only enhances text representation but also avoids word segmentation errors. Additionally, a MedBert model for learning medical knowledge is embedded in the word embedding layer, which can dynamically generate feature vectors according to different contexts, helps solve the problem of polysemy and specialized vocabulary in electronic medical records. At the same time, adding a graph attention module in the coding layer enhances the network’s ability to learn text context relationships and enhances the model’s learning of medical special grammar. Finally, F1 values of 86.38% and 84.76% are obtained on the cEHRNER and cMedQANER datasets, respectively, showing better robustness compared to other models.

Key words: graph attention networks, word embedding, named entity recognition, Bert model

摘要： 中文临床自然语言中富含大量的病历信息，对电子病历进行命名实体识别有助于建立医学辅助诊断系统，对医学领域的发展具有重要的意义，同时有利于下游任务如关系提取、建立知识图谱的实现。但中文电子病历存在中文分词困难、医学专业术语多、含有特殊表达方式的问题，易造成文本特征表达错误，于是提出基于增强词信息和图注意力的医学命名实体识别研究模型，通过增强局部特征和全局特征提高网络模型的性能。由于嵌入单一的字向量进行中文实体识别易忽略文本中词信息及语义，为此在字向量中嵌入与其高度关联的词向量，既增强文本表示，又避免分词错误的问题，并且在嵌入层中嵌入了学习医疗知识的MedBert模型，该模型能根据不同语境动态生成特征向量，有助于解决电子病历中一词多义及专业词汇的问题。同时，在编码层中添加图注意力模块增强模型学习文本上下文关系的能力和对医疗特殊语法的学习。在cEHRNER和cMedQANER数据集的实验上分别获得了86.38%和84.76%的F1值，与其他模型相比有较好的鲁棒性。

关键词: 图注意力, 匹配词, 命名实体识别, Bert模型

ZHAO Zhenzhen, DONG Yanru, LIU Jing, ZHANG Junzhong, CAO Hui. Medical Named Entity Recognition Incorporating Word Information and Graph Attention[J]. Computer Engineering and Applications, 2024, 60(11): 147-155.

赵珍珍, 董彦如, 刘静, 张俊忠, 曹慧. 融合词信息和图注意力的医学命名实体识别[J]. 计算机工程与应用, 2024, 60(11): 147-155.

References

[1] TANG B, WANG X, YAN J, et al. Entity recognition in Chinese clinical text using attention-based CNN-LSTM-CRF[J]. BMC Medical Informatics and Decision Making, 2019, 19(S3): 74.
[2] XUE M, CAI W, SU J, et al. Neural collective entity linking based on recurrent random walk network learning[J]. arXiv:1906.09320, 2019.
[3] ZHU T, QIN Y, XIANG Y, et al. Distantly supervised biomedical relation extraction using piecewise attentive convolutional neural network and reinforcement learning[J]. Journal of the American Medical Informatics Association, 2021, 28(12): 2571-2581.
[4] BARTOLO M, THRUSH T, JIA R, et al. Improving question answering model robustness with synthetic adversarial data generation[J]. arXiv:2104.08678, 2021.
[5] LIN Z H, YANG D, YIN X C. Patient similarity via joint embeddings of medical knowledge graph and medical entity descriptions[J]. IEEE Access, 2020, 8: 156663-156676.
[6] NADEAU D, SEKINE S. A survey of named entity recognition and classification[J]. Lingvisticae Investigationes, 2007, 30(1): 3-26.
[7] NADEAU D, TURNEY P D, MATWIN S. Unsupervised named-entity recognition: generating gazetteers and resolving ambiguity[C]//Proceedings of the 19th International Conference on Advances in Artificial Intelligence: Canadian Society for Computational Studies of Intelligence, 2006: 266-277.
[8] ZHENG X, DU H, LUO X, et al. BioByGANS: biomedical named entity recognition by fusing contextual and syntactic features through graph attention network in node classification framework[J]. BMC Bioinformatics, 2022, 23(1): 1-19.
[9] ZHANG Z Y, CHEN A L P. Biomedical named entity recognition with the combined feature attention and fully-shared multi-task learning[J]. BMC Bioinformatics, 2022, 23(1): 458.
[10] 崔少国, 陈俊桦, 李晓虹. 融合语义及边界信息的中文电子病历命名实体识别[J]. 电子科技大学学报, 2022, 51(4): 565-571.
CUI S G, CHEN J H, LI X H. Named entity recognition for Chinese electronic medical record by fusing semantic and boundary information[J]. Journal of University of Electronic Science and Technology of China, 2022, 51(4): 565-571.
[11] 孙振, 李新福. 多特征融合的中文电子病历命名实体识别[J]. 计算机工程与应用, 2023, 59(23): 136-144.
SUN Z, LI X F. Named entity recognition of Chinese electronic medical records based on multi-feature fusion[J]. Computer Engineering and Applications, 2023, 59(23): 136-144.
[12] 雷松泽, 刘博, 王瑜菲, 等. 结合多特征嵌入和多网络融合的中文医疗命名实体识别[J]. 电子与信息学报, 2023(8): 3032-3039.
LEI S Z, LIU B, WANG Y F, et al. Chinese medical named entity recognition combined with multi-feature embedding and multi-network fusion[J]. Journal of Electronics & Information Technology, 2023(8): 3032-3039.
[13] 封红旗, 孙杨, 杨森, 等. 基于BERT的中文电子病历命名实体识别[J]. 计算机工程与设计, 2023, 44(4): 1220-1227.
FENG H Q, SUN Y, YANG S, et al. Chinese electronic medical record named entity recognition based on BERT methods[J]. Computer Engineering and Design, 2023, 44(4): 1220-1227.
[14] ZHANG Y, YANG J. Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018: 1118-1144.
[15] MA R T, PENG M L, ZHANG Q, et al. Simplify the usage of lexicon in Chinese NER[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 5951-5960.
[16] GUI T, ZOU Y, ZHANG Q, et al. A lexicon-based graph neural network for Chinese NER[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019: 1040-1050.
[17] 陈雪松, 朱鑫海, 王浩畅. 基于PMV-LSTM的中文医学命名实体识别[J]. 计算机工程与设计, 2022, 43(11): 3257-3263.
CHEN X S, ZHU X H, WANG H C. Chinese medical named entity recognition based on PMV-LSTM[J]. Computer Engineering and Design, 2022, 43(11): 3257-3263.
[18] SI Y, WANG J, XU H, et al. Enhancing clinical concept extraction with contextual embeddings[J]. Journal of the American Medical Informatics Association, 2019, 26(11): 1297-1304.
[19] PENNINGTON J, SOCHER R, MANNING C. Glove: global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1532-1543.
[20] 罗凌, 杨志豪, 宋雅文, 等. 基于笔画ELMo和多任务学习的中文电子病历命名实体识别研究[J]. 计算机学报, 2020, 43(10): 1943-1957.
LUO L, YANG Z H, SONG Y W, et al. Chinese clinical named entity recognition based on stroke ELMo and Multi-Task learning[J]. Chinese Journal of Computers, 2020, 43(10): 1943-1957.
[21] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[22] ZHANG X, ZHANG Y, ZHANG Q, et al. Extracting comprehensive clinical information for breast cancer using deep learning methods[J]. International Journal of Medical Informatics, 2019, 132: 1-7.
[23] LI D, LONG J, QU J, et al. Chinese clinical named entity recognition with ALBERT and MHA mechanism[J]. Evidence Based Complementary and Alternative Medicine, 2022, 2022: 2056039.
[24] 张付领. 结合ERNIE2.0的医疗中文命名实体识别模型[J]. 电子设计工程, 2023, 31(4): 38-42.
ZHANG F L. Medical Chinese named entity recognition model combined with ERNIE2.0[J]. Electronic Design Engineering, 2023, 31(4): 38-42.
[25] 杨飞洪. 面向中文临床自然语言处理的BERT模型研究[D]. 北京: 北京协和医学院, 2021.
YANG F H. A research on BERT model for Chinese clinical language processing[D]. Beijing: Peking Union Medical College, 2021.
[26] DEMIR A, KOIKE-AKINO T, WANG Y, et al. EEG-GAT: graph attention networks for classification of electroencephalogram (EEG) signals[C]//Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society, 2022: 30-35.
[27] KIM J, SEO H, NASEEM M T, et al. Pathological-gait recognition using spatiotemporal graph convolutional networks and attention model[J]. Sensors, 2022, 22(13): 4863.
[28] LI Z, LIU H, ZHANG Z, et al. Learning knowledge graph embedding with heterogeneous relation attention networks[C]//Proceedings of the IEEE Transactions on Neural Networks and Learning Systems, 2022: 3961-3973.
[29] XIONG Y, PENG H, XIANG Y, et al. Leveraging multi-source knowledge for Chinese clinical named entity recognition via relational graph convolutional network[J]. Journal of Biomedical Informatics, 2022, 128: 104035.
[30] LI Y, HUI L, ZOU L, et al. Relation extraction in biomedical texts based on multi-head attention model with syntactic dependency feature: modeling study[J]. JMIR Medical Informatics, 2022, 10(10): 41136.
[31] VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks[J]. arXiv:1710.10903, 2017.
[32] ZHANG N, JIA Q, YIN K, et al. Conceptualized representation learning for Chinese biomedical text mining[J]. arXiv:2008.10813, 2020.
[33] LI X, YAN H, QIU X, et al. FLAT: Chinese NER using flat-lattice transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020: 6836-6842.
[34] ZHANG T, CAI Z, WANG C, et al. SMedBERT: a knowledge-enhanced pre-trained language model with structured semantics for medical text mining[J]. arXiv:2108.08983, 2021.
[35] ZHANG Z, HAN X, LIU Z, et al. ERNIE: enhanced language representation with informative entities[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 1441-1451.