Study of Named Entity Recognition Based on Graph Neural Network

doi:10.3778/j.issn.1002-8331.2211-0423

Abstract

Abstract: Named entity recognition is one of the pre-processing tasks in natural language processing, aiming at identifying desired entities and types from unstructured text, and is used in many downstream tasks, such as building knowledge graphs, event extraction and automatic question and answer. In recent years, with the wide application of graph neural networks in the field of natural language processing, some graph neural network-based named entity recognition methods have achieved good results. This paper systematically investigates the application of graph neural networks in named entity recognition, firstly, describes the development process of named entity recognition, secondly, introduces graph neural networks and three variant models, then details how to use the features of graph neural networks for named entity recognition tasks, and finally, this paper proposes possible directions and ideas for future research and summarizes them.

Key words: named entity recognition, graph neural network, natural language processing, deep learning

摘要： 命名实体识别是自然语言处理的预处理任务之一，目的是从非结构化文本中识别出所需的实体及类型，应用于众多下游任务，例如构建知识图谱、事件抽取及自动问答等。近几年，随着自然语言处理领域对图神经网络的广泛应用，一些基于图神经网络的命名实体识别方法取得了较好的结果。对图神经网络在命名实体识别中的应用进行了系统性的调研，描述了命名实体识别的发展进程，介绍了图神经网络及三种变体模型，详细分析了如何利用图神经网络的特点在命名实体识别任务上的应用研究，最后提出了未来可能研究的方向和思路。

关键词: 命名实体识别, 图神经网络, 自然语言处理, 深度学习

SHU Wenhao, XI Xuefeng, CUI Zhiming, GU Chenkai. Study of Named Entity Recognition Based on Graph Neural Network[J]. Computer Engineering and Applications, 2023, 59(19): 52-65.

束文豪, 奚雪峰, 崔志明, 顾晨凯. 图神经网络在命名实体识别中的应用研究[J]. 计算机工程与应用, 2023, 59(19): 52-65.

References

[1] GRISHMAN R，SUNDHEIM B M.Message understanding conference-6：a brief history[C]//Proceedings of the 16th International Conference on Computational Linguistics，1996.
[2] THIELEN C.An approach to proper name tagging for German[J].arXiv：cmp-lg/9506024，1995.
[3] LEE S，LEE G G.Heuristic methods for reducing errors of geographic named entities learned by bootstrapping[C]//Proceedings of the 2nd International Joint Conference on Natural Language Processing，2005：658-669
[4] FLEISCHMAN M，HOVY E.Fine grained classification of named entities[C]//Proceedings of the 19th International Conference on Computational Linguistics，2002.
[5] RAU L F.Extracting company names from text[C]//Proceedings of the 7th IEEE Conference on Artificial Intelligence Application，1991：29-32.
[6] 韩春燕，刘玉娇，琚生根，等.中文微博命名实体识别[J].四川大学学报（自然科学版），2015，52（3）：511-516.
HAN C Y，LIU Y J，JU S G，et al.Named entity recognition in Chinese micro-blog[J].Journal of Sichuan University（Natural Science Edition），2015，52（3）：511-516.
[7] FENG J，LI Z，ZHANG D.Bridge detection text named entity recognition based on hidden Markov model[J].Traffic World，2020，8：32-33.
[8] COLLOBERT R，WESTON J，BOTTOU L，et al.Natural language processing（almost） from scratch[J].Journal of Machine Learning Research，2011，12：2493-2537.
[9] CAO Y，ZHOU Y，SHEN F，et al.Research on named entity recognition of Chinese electronic medical records based on CNN-CRF[J].Journal of Chongqing University of Posts and Telecommunications（Natural Science Edition），2019，6：869-875.
[10] KONG J，ZHANG L，JIANG M，et al.Incorporating multi-level CNN and attention mechanism for Chinese clinical named entity recognition[J].Journal of Biomedical Informatics，2021，116：103737.
[11] HUANG Z，XU W，YU K.Bidirectional LSTM-CRF models for sequence tagging[J].arXiv：1508.01991，2015.
[12] YANG H，LI L，YANG R.Recognition model of electronic medical record named entity based on bidirectional LSTM neural network[J].Chinese Tissue Engineering Research，2018，22：3237-3242.
[13] JI X，ZHU Y，LI F.Chinese named entity recognition based on Attention-BiLSTM[J].Journal of Hunan Univ Technol，2019，5：14.
[14] LIU Y，LI D.Chinese named entity recognition method based on BLSTM-CNN-CRF[J].Journal of Harbin University of Science and Technology，2020，25（1）：115-120.
[15] YAN H，DENG B，LI X，et al.TENER：adapting transformer encoder for named entity recognition[J].arXiv：1911.04474，2019.
[16] 李博，康晓东，张华丽，等.采用Transformer-CRF的中文电子病历命名实体识别[J].计算机工程与应用，2020，56（5）：153-159.
LI B，KANG X D，ZHANG H L，et al.Named entity recognition in Chinese electronic medical records using Transformer-CRF[J].Computer Engineering and Applications，2020，56（5）：153-159.
[17] SHEN T，YU L，JIN L.Research on Chinese entity recognition based on BERT-BILSTM-CRF model[J].Journal of Qiqihar University（Natural Science Edition），2022，38（1）：26-32.
[18] SCARSELLI F，GORI M，TSOI A C，et al.The graph neural network model[J].IEEE Transactions on Neural Networks，2008，20（1）：61-80.
[19] CHO K，VAN MERRI?NBOER B，GULCEHRE C，et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[J].arXiv：1406.1078，2014.
[20] HOCHREITER S，BENGIO Y，FRASCONI P，et al.Gradient flow in recurrent nets：the difficulty of learning long-term dependencies[M].[S.l.]：Wiley-IEEE Press，2001.
[21] LI Y，TARLOW D，BROCKSCHMIDT M，et al.Gated graph sequence neural networks[J].arXiv：1511.05493，2015.
[22] LI Y，VINYALS O，DYER C，et al.Learning deep generative models of graphs[J].arXiv：1803.03324，2018.
[23] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[J].arXiv：1609.02907，2016.
[24] VELI?KOVI? P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[25] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017.
[26] CHEN C，KONG F.Enhancing entity boundary detection for better Chinese named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing（Volume 2：Short Papers），2021：20-25.
[27] GUO Q，QIU X，LIU P，et al.Star-transformer[J].arXiv：1902.09113，2019.
[28] SUI D，CHEN Y，LIU K，et al.Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：3830-3840.
[29] ZHU P，CHENG D，YANG F，et al.Improving Chinese named entity recognition by large-scale syntactic dependency graph[J].IEEE/ACM Transactions on Audio，Speech，and Language Processing，2022，30：979-991.
[30] ZHONG Q，TANG Y.Chinese named entity recognition based on gated graph neural network[C]//Proceedings of the International Conference on Knowledge Science，Engineering and Management，2021：604-613.
[31] 宋旭晖，于洪涛，李邵梅.基于图注意力网络字词融合的中文命名实体识别[J].计算机工程，2022，48（10）：298-305.
SONG X H，YU H T，LI S M.Chinese named entity recognition based on word fusion of graph attention network[J].Computer Engineering，2022，48（10）：298-305.
[32] LEE L H，LU Y.Multiple embeddings enhanced multi-graph neural networks for Chinese healthcare named entity recognition[J].IEEE Journal of Biomedical and Health Informatics，2021，25（7）：2801-2810.
[33] ZONG J，HAN J.Entity recognition of Chinese electronic medical record based on gated graph neural network[C]//Proceedings of the 14th International Conference on Measuring Technology and Mechatronics Automation（ICMTMA），2022：1208-1213.
[34] XIONG Y，PENG H，XIANG Y，et al.Leveraging multi-source knowledge for Chinese clinical named entity recognition via relational graph convolutional network[J].Journal of Biomedical Informatics，2022，128：104035.
[35] ZHAO Y，MENG K，LIU G.A multi-channel graph attention network for Chinese NER[C]//Proceedings of the International Conference on Neural Information Processing，2021：203-214.
[36] WANG Y，LU L，WU Y，et al.Polymorphic graph attention network for Chinese NER[J].Expert Systems with Applications，2022：117467.
[37] ZHANG W，LUO J，YANG K.Social media named entity recognition based on graph attention network[C]//Proceedings of the International Symposium on Computer Science and Intelligent Controls（ISCSIC），2021：127-131.
[38] CETOLI A，BRAGAGLIA S，O’HARNEY A D，et al.Graph convolutional networks for named entity recognition[J].arXiv：1709.10053，2017.
[39] CAO Y，HOU L，LI J，et al.Neural collective entity linking[J].arXiv：1811.08603，2018.
[40] JIA N，CHENG X，SU S，et al.CoGCN：combining co‐attention with graph convolutional network for entity linking with knowledge graphs[J].Expert Systems，2021，38（1）：e12606.
[41] PUJARY D，THORNE C，AZIZ W.Disease normalization with graph embeddings[C]//Proceedings of SAI Intelligent Systems Conference，2020：209-217.
[42] WU J，ZHANG R，MAO Y，et al.Dynamic graph convolutional networks for entity linking[C]//Proceedings of the Web Conference 2020，2020：1149-1159.
[43] CHEN Z，WU Y，FENG Y，et al.Integrating manifold knowledge for global entity linking with heterogeneous graphs[J].Data Intelligence，2022，4（1）：20-40.
[44] ZHANG Y.Collective entity linking models via graph neural network[EB/OL].（2019）[2022-11-02].https：//www.semanticscholar.org/paper/Collective-Entity-Linking-Models-via-Graph-Neural-Zhang/6ba946266b97e964fd664e1484fc-f1ae330173b8.
[45] KACUPAJ E，PLEPI J，SINGH K，et al.Conversational question answering over knowledge graphs with transformer and graph attention networks[J].arXiv：2104. 01569，2021.
[46] BO M，ZHANG M.Learning dynamic coherence with graph attention network for biomedical entity linking[C]//Proceedings of the International Joint Conference on Neural Networks（IJCNN），2021：1-8.
[47] MA J，LI D，CHEN Y，et al.A knowledge graph entity disambiguation method based on entity-relationship embedding and graph structure embedding[J].Computational Intelligence and Neuroscience，2021：2878189.
[48] SHAW P，MASSEY P，CHEN A，et al.Generating logical forms from graph representations of text and entities[J].arXiv：1905.08407，2019.
[49] GUI T，ZOU Y，ZHANG Q，et al.A lexicon-based graph neural network for Chinese NER[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：1040-1050.
[50] RATINOV L，ROTH D.Design challenges and misconceptions in named entity recognition[C]//Proceedings of the 13th Conference on Computational Natural Language Learning（CoNLL-2009），2009：147-155.
[51] DING R，XIE P，ZHANG X，et al.A neural multi-digraph model for Chinese NER with gazetteers[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：1462-1467.
[52] VRETINARIS A，LEI C，EFTHYMIOU V，et al.Medical entity disambiguation using graph neural networks[C]//Proceedings of the 2021 International Conference on Management of Data，2021：2310-2318.
[53] ZHANG Z，WU C，LI Z，et al.Author Name disambiguation using multiple graph attention networks[C]//Proceedings of the International Joint Conference on Neural Networks（IJCNN），2021：1-8.
[54] CHEN P，DING H，ARAKI J，et al.Explicitly capturing relations between entity mentions via graph neural networks for domain-specific named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing（Volume 2：Short Papers），2021.
[55] LOU Y，QIAN T，LI F，et al.A graph attention model for dictionary-guided named entity recognition[J].IEEE Access，2020，8：71584-71592.
[56] SETI X，WUMAIER A，YIBULAYIN T，et al.Named-entity recognition in sports field based on a character-level graph convolutional network[J].Information，2020，11（1）：30.
[57] HAISA G，ALTENBEK G.Deep Learning with word embedding improves Kazakh named-entity recognition[J].Information，2022，13（4）：180.
[58] HONG Y，LIU Y，YANG S，et al.Improving graph convolutional networks based on relation-aware attention for end-to-end relation extraction[J].IEEE Access，2020，8：51315-51323.
[59] LAI Q，ZHOU Z，LIU S.Joint entity-relation extraction via improved graph attention networks[J].Symmetry，2020，12（10）：1746.
[60] FU T J，LI P H，MA W Y.GraphRel：modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of Association for Computational Linguistics，2019：1409-1418.
[61] CARBONELL M，RIBA P，VILLEGAS M，et al.Named entity recognition and relation extraction with graph neural networks in semi structured documents[C]//Proceedings of the 25th International Conference on Pattern Recognition（ICPR），2021：9622-9627.
[62] PANG Y，ZHOU T，ZHANG Z.A joint model for Chinese medical entity and relation extraction based on graph convolutional networks[C]//Proceedings of the 3rd International Conference on Natural Language Processing（ICNLP），2021：119-124.
[63] KAMBAR M E Z N.Chemical-gene relation extraction with graph neural networks and bert encoder[C]//Proceedings of the International Conference on Innovations in Computing Research，2022：166.
[64] LI T，MA L，QIN J，et al.DTGCN：a method combining dependency tree and graph convolutional networks for Chinese long-interval named entity relationship extraction[J].Journal of Ambient Intelligence and Humanized Computing，2022，258：1-13.
[65] LUO Y，ZHAO H.Bipartite flat-graph network for nested named entity recognition[J].arXiv：2005.00436，2020.
[66] ZHOU L，LI J，GU Z，et al.PANNER：POS-aware nested named entity recognition through heterogeneous graph neural network[J].IEEE Transactions on Computational Social Systems，2022（1）.
[67] SUI Y，BU F，HU Y，et al.Trigger-GNN：a trigger-based graph neural network for nested named entity recognition[C]//Proceedings of the International Joint Conference on Neural Networks（IJCNN），2022：1-8.
[68] TRAN T T，MIWA M，ANANIADOU S.Syntactically-informed word representations from graph neural network[J].Neurocomputing，2020，413：431-443.
[69] JIN H，HOU L，LI J，et al.Fine-grained entity typing via hierarchical multi graph convolutional networks[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：4969-4978.
[70] XU L，JIE Z，LU W，et al.Better feature integration for named entity recognition[J].arXiv：2104.05316，2021.
[71] SUN X，ZHOU J，WANG S，et al.Linguistic dependency guided graph convolutional networks for named entity recognition[C]//Proceedings of the International Conference on Advanced Data Mining and Applications，2022：237-248.
[72] ZARATIANA U，TOMEH N，HOLAT P，et al.GNNER：reducing overlapping in Span-based NER using graph neural networks[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics：Student Research Workshop，2022：97-103.