Sentence Matching Based on Dependency Syntax and Graph Attention Network

doi:10.3778/j.issn.1002-8331.2112-0391

Abstract

Abstract: Sentence matching is a basic task in natural language processing, which can be applied to natural language inference, paraphrase recognition and other scenarios. At present, most of the mainstream models use attention mechanism to realize the alignment of words or phrases between two sentences. However, these models usually ignore the internal structure of sentences and do not consider the dependency between text units. To solve this problem, this paper proposes a matching model based on dependency syntax and graph attention network. Two methods are designed to model sentence pairs as semantic graph. Graph attention network is used to encode the constructed graph for sentence matching. Experimental results show that the proposed model can learn graph structure well, and the accuracy of the model on the natural language inference dataset SNLI and the paraphrase recognition dataset Quora is 88.7% and 88.9% respectively.

Key words: sentence matching, dependency syntax, graph attention network, co-occurrence semantic graph, full-connected semantic graph

摘要： 句子匹配是自然语言处理的一项基本任务，可应用于自然语言推理、释义识别等多个场景。目前，主流的模型大多采用注意力机制来实现两个句子之间单词或短语的对齐。然而，这些模型通常忽略了句子的内在结构，没有考虑文本单元之间的依存关系。针对此问题，提出了一种基于依存句法和图注意力网络的匹配模型。设计两种建模方式将句子对建模为语义图。使用图注意力网络对构建的图进行编码以进行句子匹配。实验结果表明，提出的模型可以较好地学习图结构，在自然语言推理数据集SNLI和释义识别数据集Quora上分别达到了88.7%和88.9%的准确率。

关键词: 句子匹配, 依存句法, 图注意力网络, 词共现语义图, 全连通语义图

YANG Chunxia, CHEN Qigang, XU Ben, MA Wenwen. Sentence Matching Based on Dependency Syntax and Graph Attention Network[J]. Computer Engineering and Applications, 2023, 59(9): 123-129.

杨春霞, 陈启岗, 徐奔, 马文文. 基于依存句法和图注意力网络的句子匹配[J]. 计算机工程与应用, 2023, 59(9): 123-129.

References

[1] WANG Z G，HAMZA W，FLORIAN R.Bilateral multi-perspective matching for natural language sentences[C]//Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence，2017：4144-4150.
[2] 郭茂盛，张宇，刘挺.文本蕴含关系识别与知识获取研究进展及展望[J].计算机学报，2017，40（4）：889-910.
GUO M S，ZHANG Y，LIU T.Research advances and prospect of recognizing textual entailment and knowledge acquisition[J].Chinese Journal of Computers，2017，40（4）：889-910.
[3] BOWMAN S R，ANGELI G，POTTS C，et al.A large annotated corpus for learning natural language inference[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing，2015：632-642.
[4] PARIKH A P，TACKSTR OM O，DAS D.A decomposable attention model for natural language inference[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，2016：2249-2255.
[5] CHEN Q，ZHU X D，LING Z H，et al.Enhanced LSTM for natural language inference[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics，2016：1657-1688.
[6] LIU Y，GARDNER M，LAPATA M.Structured alignmentnetworks for matching sentences[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：1554-1564.
[7] HUANG P S，HE X，GAO J，et al.Learning deep structured semantic models for web search using clickthrough data[C]//Proceedings of the 22nd ACM International Conference on Information & Knowledge Management，2013：2333-2338.
[8] QIU X，HUANG X.Convolutional neural tensor network architecture for community-based question answering[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence，2015：1305-1311.
[9] TAN C Q，WEI F R，WANG W H，et al.Multiway attention networks for modeling sentence pairs[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence，2018：4411-4417.
[10] ZHAO K，HUANG L，MA M B.Textual entailment with structured attentions and composition[C]//Proceedings of COLING 2016，the 26th International Conference on Computational Linguistics：Technical Papers，2016：2248-2258.
[11] VELICKOVIC P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[12] 方红，苏铭，冯一铂，等.结合gazetteers和句法依存树的中文命名实体识别[J].计算机工程与应用，2022，58（18）：227-232.
FANG H，SU M，FENG Y B，et al.Chinese named entity recognition with combind with gazetteers and syntactic dependency tree[J].Computer Engineering and Applications，2022，58（18）：227-232.
[13] PARIKH A P，T?CKSTR?M O，DAS D，et al.A decomposable attention model for natural language inference[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，2016：2249-2255.
[14] 姜克鑫，赵亚慧，崔荣一.融合高低层语义信息的自然语言句子匹配方法[J].计算机应用研究，2022，39（4）：1060-1063.
JIANG K X，ZHAO Y H，CUI R Y.Natural language sentence matching method fusion of high-level and low-level semantic information[J].Application Research of Computers，2022，39（4）：1060-1063.
[15] 郑德权，于凤，王贺伟.融合语言知识与深度学习的文本蕴含识别[J].计算机工程与应用，2020，56（24）：157-163.
ZHENG D Q，YU F，WANG H W.Text entailment recognition based on integration of language knowledge and deep learning[J].Computer Engineering and Applications，2020，56（24）：157-163.
[16] GONG Y，LUO H，ZHANG J.Natural language inference over interaction space[J].arXiv：1709.04348，2017.
[17] WANG Z，MI H，ITTYCHERIAH A.Sentence similarity learning by lexical decomposition and composition[J]. arXiv：1602.07019，2016.
[18] 杨德志，柯显信，余其超，等.基于RCNN的问题相似度计算方法[J].计算机工程与科学，2021，43（6）：1076-1080.
YANG D Z，KE X X，YU Q C，et al. A question similarity calculation method based on RCNN[J].Computer Engineering & Science，2021，43（6）：1076-1080.
[19] 赵琪，杜彦辉，芦天亮，等.基于Capsule-BiGRU的文本相似度分析算法[J].计算机工程与应用，2021，57（15）：171-177.
ZHAO Q，DU Y H，LU T L，et al. Algorithm of text similarity analysis based on capsule-BiGRU[J].Computer Engineering and Applications，2021，57（15）：171-177.
[20] TOMAR G S，DUQUE T，T?CKSTR?M O，et al.Neural paraphrase identification of questions with noisy pretraining[J].arXiv：1704.04565，2017.