Named Entity Recognition in Electromechanical Field Based on Data Enhancement and Loss Balancing

doi:10.3778/j.issn.1002-8331.2311-0310

Abstract

Abstract: Named entity recognition in electromechanical field is mechatronic innovation design’s most essential information retrieval process. The named entity recognition task data in the electromechanical field is small and has a data imbalance problem. In this paper, a named entity recognition dataset in the electromechanical field is constructed, a multidimensional data enhancement method is designed according to the dataset features, and a named entity recognition model BERT-BiGRU-CRF (BL) based on improved loss is proposed. Firstly, it crawls and annotates the text corpus in the field of electromechanics on the Internet to constitute the named entity recognition dataset in the field of electromechanics; then, it carries out multidimensional data enhancement according to the effects of different ways on the dataset from four aspects, nam ely, similar entity substitution, synonym substitution, corpus culling, and corpus splicing, and then it increases the data richness by a specific ratio of data expansion; finally, it designs the use of the data imbalance in the dataset. For the problem of data imbalance in the data set, Weigh loss is designed to use the model of balancing the weights of focal loss and CRF loss, which firstly adopts BERT to encode the word vectors, then uses BiGRU to complete the feature extraction of the text vectors, and finally uses CRF to perform label constraints and decoding. Experimentally, it is proved that the multidimensional data enhancement method in this paper has significantly improved the model effect, and the improved model performs optimally on both the original and the enhanced dataset, with F1 values of 78.23% and 83.3%, respectively.

Key words: electromechanical field, named entity recognition, data enhancement, focal loss, Weigh loss

摘要： 机电领域命名实体识别是机电创新设计信息检索最基础的过程。目前命名实体识别任务的数据在机电领域较少，且大部分存在不平衡问题。通过构建机电领域命名实体识别数据集，根据数据集文本结构特点设计多维数据增强方法，并提出基于改进loss的命名实体识别模型BERT-BiGRU-CRF（BL）。对互联网机电领域文本语料进行爬取并进行标注构成机电领域命名实体识别数据集；根据不同方式对数据集的影响从同类实体替换、同义词替换、语料裁减和语料拼接四个方面进行多维数据增强后按一定比例进行数据扩充增加数据丰富度；针对数据集数据不平衡问题设计使用Weigh loss平衡focal loss 与CRF loss权重的模型，该模型采用BERT进行词向量编码，利用BiGRU完成文本向量的特征提取，使用CRF进行标签约束与解码。经实验证明，多维数据增强方法对模型效果有显著提升，并且经过改进的模型在原始和增强后数据集上表现均为最优，F1值分别为78.23%和83.3%。

关键词: 机电领域, 命名实体识别, 数据增强, focal loss, Weigh loss

LIN Na, YUE Xi, TANG Dan. Named Entity Recognition in Electromechanical Field Based on Data Enhancement and Loss Balancing[J]. Computer Engineering and Applications, 2025, 61(7): 222-232.

林娜, 岳希, 唐聃. 基于数据增强和损失平衡的机电领域命名实体识别[J]. 计算机工程与应用, 2025, 61(7): 222-232.

References

[1] YAO L G, HUANG H S, WANG K W, et al. Fine-grained mechanical Chinese named entity recognition based on ALBERT-AttBiLSTM-CRF and transfer learning[J]. Symmetry, 2020, 12(12): 1986.
[2] 陈秋瑗, 程光, 李迪, 等. 机械设计领域的命名实体识别研究[J]. 计算机工程与应用, 2017, 53(20): 100-104.
CHEN Q Y, CHENG G, LI D, et al. Named entity recognition for mechanical design and manufacturing area[J]. Computer Engineering and Applications, 2017, 53(20): 100-104.
[3] NADEAU D, SEKINE S. A survey of named entity recognition and classification[J]. Lingvisticae Investigationes, 2007, 30(1): 3-26.
[4] HUANG C, WANG Y Z, YU Y Q, et al. Chinese named entity recognition of geological news based on BERT model[J]. Applied Sciences, 2022, 12(15): 7708.
[5] TAN M Y, BAO F L, GAO G L, et al. An attention-based approach for Mongolian news named entity recognition[C]//Proceedings of the 18th China National Conference on Chinese Computational Linguistics. Cham: Springer International Publishing, 2019: 424-435.
[6] WANG C Y, WANG H, ZHUANG H, et al. Chinese medical named entity recognition based on multi-granularity semantic dictionary and multimodal tree[J]. Journal of Biomedical Informatics, 2020, 111: 103583.
[7] SHI J T, SUN M X, SUN Z Y, et al. Multi-level semantic fusion network for Chinese medical named entity recognition[J]. Journal of Biomedical Informatics, 2022, 133: 104144.
[8] ZHANG Q H, HOU L, LV P T, et al. Chinese medical entity recognition model based on character and word vector fusion[J]. Scientific Programming, 2021, 2021(1): 1-12.
[9] 张智源, 孙水华, 徐诗傲, 等. 基于BERT和多窗口门控CNN的电机领域命名实体识别[J]. 计算机应用研究, 2023, 40(1): 107-114.
ZHANG Z Y, SUN S H, XU S A, et al. Named entity recognition in motor field based on BERT and multi-window gated CNN[J]. Application Research of Computers, 2023, 40(1): 107-114.
[10] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[11] YAN S, CHAI J P, WU L Y. Bidirectional GRU with multi-head attention for Chinese NER[C]//Proceedings of the 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference. Piscataway: IEEE, 2020: 1160-1164.
[12] LAFFERTY J. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning, 2001.
[13] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2980-2988.
[14] CIPOLLA R, GAL Y, KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7482-7491.
[15] ZHENG G P. Research on the recognition of Chinese named entity based on rules and statistics[J]. Information Science, 2012, 30(5): 708-712.
[16] ZHOU G D, SU J, ZHOU G D, et al. Named entity recognition using an HMM-based chunk tagger[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. New York: ACM, 2002: 473-480.
[17] LIU J G, XIA C H, YAN H H, et al. Innovative deep neural network modeling for fine-grained Chinese entity recognition[J]. Electronics, 2020, 9(6): 1001.
[18] LEE C. LSTM-CRF models for named entity recognition[J]. IEICE Transactions on Information and Systems, 2017, 100(4): 882-887.
[19] GUI T, MA R T, ZHANG Q, et al. CNN-based Chinese NER with lexicon rethinking[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, 2019: 4982-4988.
[20] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv: 1301.3781, 2013.
[21] PENNINGTON J, SOCHER R, MANNING C D. GloVe: global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2023.
[22] PETERS M E, NEUMANN M, IYYER M, et al. Deep contextualized word representations[J]. arXiv:1802.05365, 2018.
[23] 郭振东, 林民, 李成城, 等. 基于BERT-CRF的领域词向量生成研究[J]. 计算机工程与应用, 2022, 58(21): 156-162.
GUO Z D, LIN M, LI C C, et al. Research on domain-specific word vector generation based on BERT-CRF[J]. Computer Engineering and Applications, 2022, 58(21): 156-162.
[24] 康怡琳, 孙璐冰, 朱容波, 等. 深度学习中文命名实体识别研究综述[J]. 华中科技大学学报(自然科学版), 2022, 50(11): 44-53.
KANG Y L, SUN L B, ZHU R B, et al. Survey on Chinese named entity recognition with deep learning[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2022, 50(11): 44-53.
[25] 李军怀, 陈苗苗, 王怀军, 等. 基于ALBERT-BGRU-CRF的中文命名实体识别方法[J]. 计算机工程, 2022, 48(6): 89-94.
LI J H, CHEN M M, WANG H J, et al. Chinese named entity recognition method based on ALBERT-BGRU-CRF[J]. Computer Engineering, 2022, 48(6): 89-94.
[26] LV X, XIE Z, XU D X, et al. Chinese named entity recognition in the geoscience domain based on BERT[J]. Earth and Space Science, 2022, 9(3): e2021EA002166.
[27] YANG L, FU Y F, DAI Y. BIBC: a Chinese named entity recognition model for diabetes research[J]. Applied Sciences, 2021, 11(20): 9653.
[28] 马晓琴, 郭小鹤, 薛峪峰, 等. 针对命名实体识别的数据增强技术[J]. 华东师范大学学报(自然科学版), 2021, 55(5): 14-23.
MA X Q, GUO X H, XUE Y F, et al. Data augmentation technology for named entity recognition[J]. Journal of East China Normal University (Natural Science), 2021, 55(5): 14-23.
[29] 刘兴丽, 范俊杰, 马海群. 面向小样本命名实体识别的数据增强算法改进策略研究[J]. 数据分析与知识发现, 2022, 6(10): 128-141.
LIU X L, FAN J J, MA H Q. Improvement of data augment algorithm for named entity recognition with small samples[J]. Data Analysis and Knowledge Discovery, 2022, 6(10): 128-141.
[30] 李健, 张克亮, 唐亮, 等. 面向中文命名实体识别任务的数据增强[J]. 计算机与现代化, 2022(4): 1-6.
LI J, ZHANG K L, TANG L, et al. Data augmentation for Chinese named entity recognition task[J]. Computer and Modernization, 2022(4): 1-6.
[31] 杨鹤, 于红, 刘巨升, 等. 基于BERT+BiLSTM+CRF深度学习模型和多元组合数据增广的渔业标准命名实体识别[J]. 大连海洋大学学报, 2021, 36(4): 661-669.
YANG H, YU H, LIU J S, et al. Fishery standard named entity recognition based on BERT+BiLSTM+CRF deep learning model and multivariate combination data augmentation[J]. Journal of Dalian Ocean University, 2021, 36(4): 661-669.
[32] 王蓬辉, 李明正, 李思. 基于数据增强的中文医疗命名实体识别[J]. 北京邮电大学学报, 2020, 43(5): 84-90.
WANG P H, LI M Z, LI S. Data augmentation for Chinese clinical named entity recognition[J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5): 84-90.
[33] JEAN C. Assessing agreement on classification tasks: the Kappa statistic[J]. Computational Linguistics, 1996, 22(2): 249-254.
[34] FALOTICO R, QUATTO P. Fleiss’ Kappa statistic without paradoxes[J]. Quality & Quantity, 2015, 49(2): 463-470.
[35] WEI J, ZOU K. EDA: easy data augmentation techniques for boosting performance on text classification tasks[J]. arXiv:1901.11196, 2019.
[36] LIU S, YANG H, LI J Y, et al. Chinese named entity recognition method in history and culture field based on BERT[J]. International Journal of Computational Intelligence Systems, 2021, 14(1): 1-10.
[37] 许力, 李建华. 基于BERT和BiLSTM-CRF的生物医学命名实体识别[J]. 计算机工程与科学, 2021, 43(10): 1873-1879.
XU L, LI J H. Biomedical named entity recognition based on BERT and BiLSTM-CRF[J]. Computer Engineering & Science, 2021, 43(10): 1873-1879.
[38] SUN J L, LIU Y R, CUI J, et al. Deep learning-based methods for natural hazard named entity recognition[J]. Scientific Reports, 2022, 12(1): 4598.
[39] LIPTON Z C, BERKOWITZ J, ELKAN C, et al. A critical review of recurrent neural networks for sequence learning[J]. arXiv:1506.00019, 2015.
[40] CHAI S Z, YANG Z Y, LV C S, et al. An end-to-end model based on TDNN-BiGRU for keyword spotting[C]//Proceedings of the 2019 International Conference on Asian Language Processing. Piscataway: IEEE, 2019: 402-406.
[41] LIU M Y, TU Z Y, ZHANG T, et al. LTP: a new active learning strategy for CRF-based named entity recognition[J]. Neural Processing Letters, 2022, 54(3): 2433-2454.
[42] 雷迪. 面向中医药知识图谱的命名实体识别及关系抽取[D]. 石家庄: 河北地质大学, 2021: 23-26.
LEI D. Named entity recognition and relation extraction for traditional Chinese medicine knowledge graph[D]. Shijiazhuang: Hebei GEO University, 2021: 23-26.
[43] JADON S. A survey of loss functions for semantic segmentation[C]//Proceedings of the 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology. Piscataway: IEEE, 2020: 1-7.
[44] QIN Q L, ZHAO S, LIU C M. A BERT-BiGRU-CRF model for entity recognition of Chinese electronic medical records[J]. Complexity, 2021, 2021(1): 1-11.
[45] 李妮, 关焕梅, 杨飘, 等. 基于BERT-IDCNN-CRF的中文命名实体识别方法[J]. 山东大学学报(理学版), 2020, 55(1): 102-109.
LI N, GUAN H M, YANG P, et al. BERT-IDCNN-CRF for named entity recognition in Chinese[J]. Journal of Shandong University (Natural Science), 2020, 55(1): 102-109.
[46] GAO D, PENG L F, BAI Y J. HAZOP text named entity recognition using CNN-BilSTM-CRF model[C]//Proceedings of the 2020 Chinese Automation Congress. Piscataway: IEEE, 2020: 6159-6164.
[47] ZHANG Y, YANG J. Chinese NER using lattice LSTM[J]. arXiv:1805.02023, 2018.
[48] MENG F Q, YANG S S, WANG J D, et al. Creating knowledge graph of electric power equipment faults based on BERT-BiLSTM-CRF model[J]. Journal of Electrical Engineering & Technology, 2022, 17(4): 2507-2516.