融合ERNIE与知识增强的临床短文本分类研究

doi:10.3778/j.issn.1002-8331.2401-0002

摘要/Abstract

摘要： 提出一种引入专业医疗知识与文本独特知识的ERNIE模型用于识别无规则的临床短文本。目前ERNIE模型具有一定的医疗领域知识，但是在处理下游任务时无法引入专业医疗知识与文本独特知识，因此为提高临床短文本分类的精确度与效率，提出KW-ERNIE-BiGRU模型。该模型通过引入医学知识与文本独特知识的ERNIE模型训练文本的特征向量，利用BiGRU强化上下文信息，最终在输出层进行文本分类。通过在真实的临床文本的验证与对比实验，KW-ERNIE-BiGRU模型的精确率、召回率、宏F1分别为93.4%、92.1%、92.7%，均优于其他模型。

关键词: 深度学习, 知识图谱, ERNIE, 语义强化, 临床短文本分类

Abstract: This paper proposes an ERNIE model that incorporates professional medical knowledge and unique textual knowledge for identifying irregular clinical short texts. At present, the ERNIE model has certain knowledge in the medical field, but it cannot introduce professional medical knowledge and unique textual knowledge when processing downstream tasks. Therefore, the paper proposes the KW-ERNIE-BiGRU model to improve the accuracy and efficiency of clinical short text classification. The model trains text feature vectors by introducing medical knowledge and unique textual knowledge into the ERNIE model, uses BiGRU to capture contextual information, and finally performs text classification in the output layer. Through verification and comparative experiments in real clinical texts, the accuracy, recall, and macro F1 of the KW-ERNIE-BiGRU model are 93.4%, 92.1%, and 92.7%, respectively, which are superior to other models.

Key words: deep learning, graph knowledge, ERNIE, semantic reinforcement, clinical short text classification

温浩, 杨洋. 融合ERNIE与知识增强的临床短文本分类研究[J]. 计算机工程与应用, 2025, 61(8): 108-116.

WEN Hao, YANG Yang. Research on Clinical Short Text Classification by Integrating ERNIE with Knowledge Enhancement[J]. Computer Engineering and Applications, 2025, 61(8): 108-116.

参考文献

[1] 许颖惠. 医院信息化背景下医疗数据的整合与应用研究[J]. 网络安全和信息化, 2023(9): 86-89.
XU Y H. Research on the integration and application of medical data under the background of hospital informatization[J]. Security & Informatization, 2023(9): 86-89.
[2] 叶琳, 罗铁清. 医疗数据治理综述[J]. 计算机时代, 2021(5): 10-12.
YE L, LUO T Q. Overview of medical data governance[J]. Computer Era, 2021(5): 10-12.
[3] MINAEE S, BOYKOV Y, PORIKLI F, et al. Image segmentation using deep learning: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(7): 3523-3542.
[4] 陈裔. 大数据分析在医疗信息安全中的角色与挑战[J]. 中国信息化, 2023(11): 74-75.
CHEN Y. The role and challenges of big data analysis in medical information security[J]. Zhongguo Xinxihua, 2023(11): 74-75.
[5] 杜永兴, 孙彤彤, 周李涌, 等. 基于Vocab-GCN的中文医疗文本分类方法[J]. 传感器与微系统, 2023, 42(8): 152-156.
DU Y X, SUN T T, ZHOU L Y, et al. Chinese medical text classification method based on Vocab-GCN[J]. Transducer and Microsystem Technologies, 2023, 42(8): 152-156.
[6] 王东梅. 电子化背景下病历档案管理的服务转型[J]. 档案管理, 2020(6): 124-125.
WANG D M. Service transformation of medical record archives management under the background of electronicization[J]. Archives Management, 2020(6): 124-125.
[7] 张强强, 苏变萍, 李敏. 基于卷积神经网络的SVM疾病症状分类模型[J]. 传感器与微系统, 2020, 39(3): 64-67.
ZHANG Q Q, SU B P, LI M. Disease symptom classification model of SVM based on CNN[J]. Transducer and Microsystem Technologies, 2020, 39(3): 64-67.
[8] MIAO K, HE X, YU J, et al. A study of Chinese news headline classification based on keyword feature expansion[J]. International Journal of Computational Intelligence Systems, 2023, 16: 71.
[9] SUN B, ZHAO P. Feature extension for Chinese short text classification based on topical N-Grams[C]//Proceedings of the 16th International Conference on Computer and Information Science, 2017: 477-482.
[10] FIGUEROA R L, FLORES C A. Extracting information from electronic medical records to identify the obesity status of a patient based on comorbidities and bodyweight measures[J]. Journal of Medical Systems, 2016, 40(8): 1-9.
[11] ONAN A. Biomedical text categorization based on ensemble pruning and optimized topic modelling[J]. Computational and Mathematical Methods in Medicine, 2018, 2018: 1-22.
[12] FLORES C A, FIGUEROA R L, PEZOA J E. Active learning for biomedical text classification based on automatically generated regular expressions[J]. IEEE Access, 2021, 9: 38767-38777.
[13] 张莎莎. 基于机器学习的心血管疾病诊断模型关键技术研究[D]. 北京: 北京邮电大学, 2023.
ZHANG S S. Research on key technologies of cardiovascular disease diagnosis models based on machine learning[D]. Beijing: Beijing University of Posts and Telecommunications, 2023.
[14] 李强, 李瑶坤, 夏书月, 等. 一种改进的医疗文本分类模型: LS-GRU[J]. 东北大学学报 (自然科学版), 2020, 41(7): 938-942.
LI Q, LI Y K, XIA S Y, et al. An improved medical text classification model: LS-GRU[J]. Journal of Northeastern University (Natural Science), 2020, 41(7): 938-942.
[15] 许浪, 李代伟, 张海清, 等. 基于神经网络的医疗文本分类研究[J]. 计算机工程与科学, 2023, 45(6): 1116-1122.
XU L, LI D W, ZHANG H Q, et al. Medical text classification based on neural network[J]. Computer Engineering and Science, 2023, 45(6): 1116-1122.
[16] IBRAHIM M A, KHAN M U G, MEHMOOD F, et al. GHS-NET a generic hybridized shallow neural network for multi-label biomedical text classification[J]. Journal of Biomedical Informatics, 2021, 116: 103699.
[17] 白思萌, 牛振东, 何慧, 等. 基于超图注意力网络的生物医学文本分类方法[J]. 数据分析与知识发现, 2022, 6(11): 13-24.
BAI S M, NIU Z D, HE H, et al. Biomedical text classification method based on hypergraph attention network[J]. Data Analysis and Knowledge Discovery, 2022, 6(11): 13-24.
[18] DRURY B, OLIVEIRA H G, LOPES D A A. A survey of the extraction and applications of causal relations[J]. Natural Language Engineering, 2022, 28(3): 361-400.
[19] LIAO W, ZENG B, YIN X, et al. An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa[J]. Applied Intelligence, 2021, 51: 3522-3533.
[20] 周浩然, 郑建立. 基于改进MC-BERT的ICD编码映射方法研究[J]. 智能计算机与应用, 2023, 13(6): 184-188.
ZHOU H R, ZHENG J L. Research on ICD code mapping method based on improved MC-BERT[J]. Intelligent Computer and Applications, 2023, 13(6): 184-188.
[21] GAO L, DAI Z, CHEN T, et al. Complement lexical retrieval model with semantic residual embeddings[C]//Proceedings of the 43rd European Conference on Information Retrieval Research, 2021: 146-160.
[22] 郑海燕. 基于多重启发式规则的英文特征值提取算法研究[J]. 自动化技术与应用, 2023, 42(11): 95-97.
ZHENG H Y. Research on English eigenvalue extraction algorithm based on multi restart rules[J]. Techniques of Automation and Applications, 2023, 42(11): 95-97.
[23] LIU W, ZHOU P, ZHAO Z, et al. K-BERT: enabling language representation with knowledge graph[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 2901-2908.
[24] WU X, DUAN J, PAN Y, et al. Medical knowledge graph: data sources, construction, reasoning, and applications[J]. Big Data Mining and Analytics, 2023, 6(2): 201-217.
[25] REAM D, SUAREZ V, TOURGEMAN I. A-135 the interceding effects of social cognition and “Cold” cognition in emotional functioning Post-TBI[J]. Archives of Clinical Neuropsychology, 2020, 35(6): 928.
[26] LEE H, YOON J, HWANG B, et al. KorealBERT: pretraining a lite BERT model for Korean language understanding[C]//Proceedings of the 25th International Conference on Pattern Recognition, 2021: 5551-5557.
[27] 吴国栋, 刘涵伟, 何章伟, 等. 知识图谱补全技术研究综述[J]. 小型微型计算机系统, 2023, 44(3): 471-482.
WU G D, LIU H W, HE Z W, et al. Review of knowledge graph completion technology[J]. Journal of Chinese Computer Systems, 2023, 44(3): 471-482.
[28] WANG Y, WANG Z, ZHANG H, et al. Microblog retrieval based on concept?enhanced pre?training model[J]. ACM Transactions on Knowledge Discovery from Data, 2023, 17(3): 1-32.
[29] 安波, 龙从军. 基于预训练语言模型的藏文文本分类[J]. 中文信息学报, 2022, 36(12): 85-93.
AN B, LONG C J. Pre-trained language model based Tibetan text classification[J]. Journal of Chinese Information Processing, 2022, 36(12): 85-93.
[30] 孙红, 黄雪阳, 徐广辉, 等. 融合掩码机制的图卷积文本分类模型[J]. 中文信息学报, 2023, 37(9): 98-107.
SUN H, HUANG X Y, XU G H, et al. Mask mechanism based graph convolution network for text classification[J]. Journal of Chinese Information Processing, 2023, 37(9): 98-107.
[31] AL-KAHTANI M S, MEHMOOD Z, SADAD T, et al. Intrusion detection in the Internet of Things using fusion of GRU-LSTM deep learning model[J]. Intelligent Automation & Soft Computing, 2023, 37(2): 2280-2290.
[32] 李雪芹, 杨文丽, 李娜娜. 基于对抗双向GRU网络的跨语言情感分类方法[J]. 计算机应用与软件, 2024, 41(1): 82-88.
LI X Q, YANG W L, LI N N. Cross linguistic sentiment classification method based on adversarial bidirectional GRU network[J]. Computer Applications and Software, 2024, 41(1): 82-88.
[33] 邢开颜, 陈文. 基于反向标签传播的多生成器主动学习算法及其在离群点检测中的应用研究[J]. 计算机科学, 2024, 51(4): 359-365.
XING K Y, CHEN W. Multi generator active learning algorithm based on reverse label propagation and its application in outlier detection[J]. Computer Science, 2024, 51(4): 359-365.