Computer Engineering and Applications ›› 2024, Vol. 60 ›› Issue (15): 161-169.DOI: 10.3778/j.issn.1002-8331.2305-0371

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Chinese Medical Named Entity Recognition Based on Multi-Layer Dynamic Fusion

LIN Lingde, LIU Na, XU Zhenshun, LI Ang, LI Chen   

  1. 1.College of Computer Science and Engineering, North Minzu University, Yinchuan 750021, China
    2.The Key Laboratory of Images & Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan 750021, China
  • Online:2024-08-01 Published:2024-07-30

基于多层动态融合的中文医疗命名实体识别

林令德,刘纳,徐贞顺,李昂,李晨   

  1. 1.北方民族大学 计算机科学与工程学院,银川 750021
    2.北方民族大学 图像图形智能处理国家民委重点实验室,银川 750021

Abstract: Aiming at the named entity recognition method based on the pre-training model, which only uses the hidden state of the last layer of the pre-training model, and ignores the problem that each Transformer layer corresponds to different text information, a multi-layer dynamic fusion method of the pre-training model is proposed. The pre-training model is used for feature extraction to obtain the hidden state sequence of each layer of the model. The hidden state information of each layer is combined through a multi-layer dynamic fusion method, which is used as the final output of the pre-training model. The conditional random field is used to process the sequence decode and complete sequence annotation. The multi-layer dynamic fusion method can make full use of the knowledge of each layer of the pre-trained model, so that the result contains rich feature information such as syntax and semantics, improves the representation ability of the model in the task, and enhances the flexibility of the model. Through experimental verification on medical text datasets CMeEE, CCKS2017 and general domain datasets Resume, Weibo, the results prove that adding multi-layer dynamic fusion method can effectively improve the effect of named entity recognition.

Key words: medical text mining, named entity recognition, pre-trained language model, multi-layer dynamic fusion

摘要: 针对基于预训练模型的命名实体识别方法仅使用了预训练模型最后一层隐状态,忽略了各Transformer层对应不同文本信息的问题,提出一种预训练模型多层动态融合方法。采用预训练模型进行特征提取,获得模型各层隐状态序列;通过多层动态融合方法对各层隐状态信息进行结合,作为预训练模型最终输出;采用条件随机场对序列进行解码,完成序列标注。多层动态融合方法可以充分利用预训练模型各层知识,使结果中包含丰富的句法、语义等特征信息,提升模型在任务中的表示能力,增强模型灵活性。通过对医疗文本数据集CMeEE、CCKS2017与通用领域数据集Resume、Weibo进行实验验证,结果证明,加入多层动态融合方法可以有效地提升命名实体识别效果。

关键词: 医疗文本挖掘, 命名实体识别, 预训练语言模型, 多层动态融合