Word Sense Disambiguation Combining Knowledge Graph and Text Hierarchical

doi:10.3778/j.issn.1002-8331.2205-0400

Abstract

Abstract: The current supervised word sense disambiguation model that utilizes annotated information with different word sense and pre-trained language models has achieved high disambiguation results. However, the supervised word sense disambiguation models are less scalable due to the difficulty of obtaining semantic data for manual annotation. The article proposes a bi-encoder word sense disambiguation method combining knowledge graph and text hierarchy, by introducing structured knowledge in the knowledge graph to supplement more extended semantic information, using the hierarchy of contextual input text to describe the meaning of words and phrases, and constructing a BERT-based bi-encoder, introducing a graph attention network to reduce the noise information in the contextual input text, so as to improve the disambiguation accuracy of the target words in phrase form, and ultimately improve the disambiguation effectiveness of the method. By comparing the method with the latest nine comparison algorithms in five test datasets, the disambiguation accuracy of the method mostly outperforms the comparison algorithms and achieves better results.

Key words: word sense disambiguation, knowledge graph, BERT, graph attention network

摘要： 当前带监督的词义消歧模型利用不同词义的注释信息和预训练的语言模型已经得到了较高的消歧结果。但是带监督的词义消歧模型的语义数据需要人工手动标注，使得带监督的词义消歧模型的扩展性较差。提出了一种结合知识图谱和文本层次结构的双编码器词义消歧方法，通过引入知识图谱中的结构化知识以补充更多的扩展语义信息，采用上下文输入文本的层次结构描述单词和短语的含义，并构筑基于BERT的双编码器，引入图注意力机制来降低上下文输入文本中的噪声信息，从而提高短语形式目标词的消歧准确率，最终提高方法的消歧效果。通过在5个测试数据集中与最新的9个对比算法的对比，该方法的消歧准确率大都优于对比算法，取得了较好的效果。

关键词: 词义消歧, 知识图谱, BERT, 图注意力机制

CAO Yukun, JIN Chengkun, TANG Yijia, WEI Ziyue, LI Yunfeng. Word Sense Disambiguation Combining Knowledge Graph and Text Hierarchical[J]. Computer Engineering and Applications, 2023, 59(14): 158-165.

曹渝昆, 金成坤, 唐艺嘉, 魏子越, 李云峰. 结合知识图谱和文本层次结构的词义消歧方法[J]. 计算机工程与应用, 2023, 59(14): 158-165.

References

[1] NAVIGLI R.Word sense disambiguation：a survey[J].ACM Computing Surveys，2009，41（2）：1-69.
[2] PILEHVAR M T，NAVIGLI R.A large-scale pseudoword-based evaluation framework for state-of-the-art word sense disambiguation[J].Computational Linguistics，2014，40（4）：837-881.
[3] LOUREIRO D，JORGE A.Language modelling makes sense：propagating representations through WordNet for full-coverage word sense disambiguation[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：5682-5691.
[4] HUANG L，SUN C，QIU X，et al.GlossBERT：BERT for word sense disambiguation with gloss knowledge[C]//Empirical Methods in Natural Language Processing，2019：3509-3514.
[5] MENG F.Word sense disambiguation based on graph and knowledge base[C]//4th EAI International Conference on Robotic Sensor Networks.Cham：Springer，2022：31-41.
[6] ALMOUSA M，BENLAMRI R，KHOURY R.A novel word sense disambiguation approach using WordNet knowledge graph[J].Computer Speech & Language，2022，74：101337.
[7] MILLER G A，LEACOCK C，TENGI R，et al.A semantic concordance[C]//Human Language Technology：Proceedings of a Workshop Held at Plainsboro，New Jersey，March 21-24，1993.
[8] PRADHAN S，LOPER E，DLIGACH D，et al.Semeval-2007 task-17：English lexical sample，SRL and all words[C]//Proceedings of the 4th International Workshop on Semantic Evaluations（SemEval-2007），2007：87-92.
[9] SNYDER B，PALMER M.The English all-words task[C]//Proceedings of SENSEVAL-3，the 3rd International Workshop on the Evaluation of Systems for the Semantic Analysis of Text，2004：41-43.
[10] NAVIGLI R，JURGENS D，VANNELLA D.Semeval-2013 task 12：multilingual word sense disambiguation[C]//Second Joint Conference on Lexical and Computational Semantics（*SEM），Volume 2：Proceedings of the Seventh International Workshop on Semantic Evaluation（SemEval 2013），2013：222-231.
[11] MORO A，NAVIGLI R.Semeval-2015 task 13：multilingual all-words sense disambiguation and entity linking[C]//Proceedings of the 9th International Workshop on Semantic Evaluation（SemEval 2015），2015：288-297.
[12] LUO F，LIU T，HE Z，et al.Leveraging gloss knowledge in neural word sense disambiguation by hierarchical co-attention[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：1402-1411.
[13] LUO F，LIU T，XIA Q，et al.Incorporating glosses into neural word sense disambiguation[C]//Meeting of the Association for Computational Linguistics，2018：2473-2482.
[14] KUMAR S，JAT S，SAXENA K，et al.Zero-shot word sense disambiguation using sense definition embeddings[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：5670-5681.
[15] HADIWINOTO C，NG H T，GAN W C.Improved word sense disambiguation using pre-trained contextualized word representations[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：5297-5306.
[16] BLEVINS T，ZETTLEMOYER L.Moving down the long tail of word sense disambiguation with gloss-informed biencoders[C]//Proceedings of the 58th Annual Meeting of the Association for Computational，2020：1006-1017.
[17] KENTON J D M W C，TOUTANOVA L K.BERT：pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of NAACL-HLT，2019：4171-4186.
[18] LAGLER K，SCHINDELEGGER M，B?HM J，et al.GPT2：empirical slant delay model for radio space geodetic techniques[J].Geophysical Research Letters，2013，40（6）：1069-1073.