Named Entity Recognition of Chinese Electronic Medical Records Based on Multi-Feature Fusion

doi:10.3778/j.issn.1002-8331.2207-0455

Abstract

Abstract: Named entity recognition is a basic task in natural language processing. Currently, the research on named entity recognition in Chinese electronic medical records does not consider the complex structure of medical texts and the uneven distribution of entity types in data sets. It only migrates the named entity recognition model from the general field to the medical field, and the recognition effect is not good. Aiming at the above problems, this paper proposes a multi-feature fusion named entity recognition model for Chinese electronic medical records. Firstly, the characters, radicals, and quadrilateral vectors are obtained to enrich the semantic representation of medical texts through Chinese characters. Secondly, the entity label labeling strategy is used to label the entity types in the vector to enhance the model’s learning of different text data types. Finally, the fusion vector is fed into the Mogrifier GRU layer to strengthen the relationship between feature representation semantics further, and CRF is used to establish label constraints. The experimental results show that the F1 value of the proposed model reaches 88.72% on the CCKS2019 dataset and 95.44% on the MSRA dataset, which verifies the effectiveness of the model.

Key words: electronic medical records, named entity recognition, multi-feature, gated recurrent unit（GRU）

摘要： 命名实体识别是自然语言处理中的基本任务。目前中文电子病历命名实体识别研究没有考虑到医疗文本结构复杂、数据集实体类型分布不均衡的情况，仅将通用领域的命名实体识别模型迁移到医疗领域，识别效果不佳。针对以上问题，提出多特征融合的中文电子病历命名实体识别模型。分别获取字、部首和四角向量，通过汉字字形丰富医疗文本的语义表示；利用实体标签标记策略，将向量中可能存在的实体类型进行标记，加强模型对不同类型文本数据的学习；将融合向量送入到Mogrifier GRU层，进一步加强特征表示语义间的联系，并利用CRF建立标签约束。实验表明，所提模型在CCKS2019数据集上的F1值达到88.72%，在MSRA数据集上达到95.44%，验证了模型的有效性。

关键词: 电子病历, 命名实体识别, 多特征, 门控循环单元（GRU）

SUN Zhen, LI Xinfu. Named Entity Recognition of Chinese Electronic Medical Records Based on Multi-Feature Fusion[J]. Computer Engineering and Applications, 2023, 59(23): 136-144.

孙振, 李新福. 多特征融合的中文电子病历命名实体识别[J]. 计算机工程与应用, 2023, 59(23): 136-144.

References

[1] 赵丹丹，黄德根，孟佳娜，等.多头注意力与字词融合的中文命名实体识别[J].计算机工程与应用，2022，58（7）：142-149.
ZHAO D D，HUANG D G，MENG J N，et al.Chinese named entity recognition by intergrating multi-heads attention mechanism and character and words fusion[J].Computer Engineering and Applications，2022，58（7）：142-149.
[2] KRAUS S，BLAKE C，WEST S L.Information extraction from medical notes[J].Opening Schools for All，2007（13）：95-103.
[3] BUNDSCHUS M，DEJORI M，STETTER M，et al.Extraction of semantic biomedical relations from text using conditional random fields[J].BMC Bioinformatics，2008，9（1）：1-14.
[4] YANG X，HUANG W.A conditional random fields approach to clinical name entity recognition[C]//Proceedings of CCKS Tasks，2018：1-6.
[5] COLLOBERT R，WESTON J，BOTTOU L，et al.Natural language processing（almost） from scratch[J].Journal of Machine Learning Research，2011，12：2493-2537.
[6] HUANG Z，XU W，YU K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv：1508.01991，2015.
[7] 唐国强，高大启，阮彤，等.融入语言模型和注意力机制的临床电子病历命名实体识别[J].计算机科学，2020，47（3）：211-216.
TANG G Q，GAO D Q，RUAN T，et al.Named entity recognition of clinical electronic medical records with language model and attention mechanism[J].Journal of Computer Science，2020，47（3）：211-216.
[8] TANG B，WANG X，YAN J.Entity recognition in Chinese clinical text using attention-based CNN-LSTM-CRF[J].BMC Medical Informatics and Decision Making，2019，19（S3）：74-82.
[9] JI B，LIU R，LI S S，et al.A BiLSTM-CRF method to Chinese electronic medical record named entity recognition[C]//Proceedings of the 2018 International Conference on Algorithms，Computing and Artificial Intelligence，2018：1-6.
[10] 李博，康晓东，张华丽，等.采用Transformer-CRF的中文电子病历命名实体识别[J].计算机工程与应用，2020，56（5）：153-159.
LI B，KANG X D，ZHANG H L，et al.Named entity recognition in Chinese electronic medical records using transformer-CRF[J].Computer Engineering and Applications，2020，56（5）：153-159.
[11] CHUNG J，GULCEHRE C，CHO K H，et al.Empirical evaluation of gated recurrent neural networks on sequence modeling[J].arXiv：1412.3555，2014.
[12] SULEA O M，NISIOI S S，DINU L P.Vsing word embeddings to translate named entities[J].Angewandte Chemie International Edition，2016，10（10）：450-459.
[13] 张云秋，汪洋，李博诚.基于RoBERTa-wwm动态融合模型的中文电子病历命名实体识别[J].数据分析与知识发现，2022，6（2）：242-250.
ZHANG Y Q，WANG Y，LI B C.Identifying named entities of Chinese electronic medical records based on RoBERTa-wwm dynamic fusion model[J].Data Analysis and Knowledge Discovery，222，6（2）：242-250.
[14] ZHANG Y，YANG J.Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，2018：1554-1564.
[15] MA R，PENG M，ZHANG Q，et al.Simplify the usage of lexicon in Chinese NER[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：5951-5960.
[16] XU C，WANG F，HAN J，et al.Exploiting multiple embeddings for Chinese named entity recognition[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management，2019：2269-2272.
[17] LAN Z，CHEN M，GOODMAN S，et al.ALBERT：a lite BERT for self-supervised learning of language representations[J].arXiv：1909.11942，2019.
[18] KENTON L，JACOB D，CHANG M W，et al.BERT：pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT，2019：4171-4186.
[19] WEI Z，SU J，WANG Y，et al.A novel cascade binary tagging framework for relational triple extraction[J].arXiv：1909.03227，2019.
[20] MELIS G，KO?ISKY T，BLUNSOM P.Mogrifier LSTM[J].arXiv：1909.01792，2019.
[21] CUI Y，CHE W，LIU T，et al.Revisiting pre-trained models for Chinese natural language processing[C]//Findings of the Association for Computational Linguistics：EMNLP 2020，2020：657-668.
[22] LI X，YAN H，QIU X，et al.FLAT：Chinese NER using flat-lattice transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics（ACL 2020），2020：6836-6842.
[23] LI N，LUO L，DING Z，et al.DUTIR at the CCKS-2019 Task1：improving Chinese clinical named entity recognition using stroke ELMo and transfer learning[C]//Proceedings of the 4th China Conference on Knowledge Graph and Semantic Computing（CCKS 2019），2019：24-27.
[24] 乔锐，杨笑然，黄文亢.基于BERT与模型融合的医疗命名实体识别[C]//2019年全国知识图谱与语义计算大会，2019：1-6.
QIAO R，YANG X R，HUANG W K.Medical named entity recognition based on BERT and model fusion[C]//China Conference on Knowledge Graph and Semantic Computing（CCKS 2019），2019：1-6.
[25] WU S，SONG X，FENG Z.MECT：multi-metadata embedding based cross-transformer for Chinese named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing（ACL 2021），2021：1529-1539.
[26] LIU W，XU T，XU Q，et al.An encoding strategy based word-character LSTM for Chinese NER[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2019：2379-2389.
[27] SUI D，CHEN Y，LIU K，et al.Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：3830-3840.
[28] GUI T，MA R，ZHANG Q，et al.CNN-based Chinese NER with lexicon rethinking[C]//International Joint Conference on Artificial Intelligence（IJCAL），2019：4982-4988.