Triple Extraction of Combining Dependency Analysis and Graph Attention Network

doi:10.3778/j.issn.1002-8331.2203-0578

Abstract

Abstract: Traditional methods of triplet extraction follow a mode of pipeline to carry out named entity recognition and relationship extraction in stages, which leads to the accuracy of entity recognition, directly affects the effect of relationship extraction, resulting in the lack of sentence context information, entity relationship overlap and other problems. Therefore, a triple joint extraction model combining dependency analysis, graph attention network and adversarial training are proposed. Firstly, the model inputs the sentence into the BiLSTM layer to extract the word features, uses the learnable linear unit to strengthen the features, and inputs the sentence into the constraint matrix generated by the syntactic analysis layer. The strengthened word features and dependency constraint matrix are input into the graph attention network to extract the sentence sequence features and the local dependency features of words, and the graph attention coefficient is calculated together. Then the sigmoid layer is used to predict the entity and entity relationship in the sentence. Finally, adversarial training is added to the word embedding layer to improve the robustness of the model. The experiment uses the public data set NYT to verify the accuracy of the model in extracting triples, and the recall rate is also significantly improved. Compared with the existing pipeline and joint methods, it improves the problems of error accumulation and relationship overlap.

Key words: knowledge graph, triple joint extraction, graph attention network, dependency analysis, adversarial training

摘要： 传统的三元组抽取采用流水线方式分阶段进行命名实体识别和关系抽取，导致实体识别的精度直接影响关系抽取的效果，造成句子上下文信息缺失，以及实体关系重叠问题等。为此，提出了结合依存分析、图注意力网络和对抗训练的三元组联合抽取模型，该模型将句子输入到BiLSTM层提取单词特征，利用可学习的线性单元进行特征强化，同时将句子输入到句法分析层生成的约束矩阵；将强化后的单词特征与依存约束矩阵输入到图注意力网络提取句子序列特征和单词的局部依赖特征，共同计算图注意力系数；再使用Sigmoid层预测出句子中的实体和实体关系；在词嵌入层加入对抗训练改善模型鲁棒性。实验采用公共数据集NYT验证了模型抽取三元组的准确率，同时召回率也显著提升，与现有的流水线和联合方法相比，改善了误差累积、关系重叠问题。

关键词: 知识图谱, 三元组联合抽取, 图注意力网络, 依存分析, 对抗训练

ZHAI Sheping, BAI Xiaoxia, ZHANG Yuhang, CHENG Dabao. Triple Extraction of Combining Dependency Analysis and Graph Attention Network[J]. Computer Engineering and Applications, 2023, 59(12): 148-156.

翟社平, 柏晓夏, 张宇航, 成大宝. 融合依存分析和图注意网络的三元组抽取[J]. 计算机工程与应用, 2023, 59(12): 148-156.

References

[1] 鄂海红，张文静，肖思琪.深度学习实体关系抽取研究综述[J].软件学报，2019，30（6）：1793-1818.
E H H，ZHANG W J，XIAO S Q.Survey of relationship extraction based on deep learning[J].Journal of Software，2019，30（6）：1793-1818.
[2] 李冬梅，张扬，李东远，等.实体关系抽取方法研究综述[J].计算机研究与发展，2020，57（7）：1424-1448.
LI D M，ZHANG Y，LI D Y，et al.Review of entity relation extraction methods[J].Journal of Computer Research and Development，2020，57（7）：1424-1448.
[3] ZHENG S C，HAO Y X，LU D Y，et al.Joint entity and relation extraction based on a hybrid neural network[J].Neurocomputing，2017，257（12）：59-66.
[4] VELIKOVI P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[5] SZEGEDY C，ZAREMBA W，SUTSKEVER I，et al.Intriguing properties of neural networks[C]//Proceedings of the 2nd International Conference on Learning Representations，2014.
[6] GOODFELLOW I J，SHLENS J，SZEGEDY C.Explaining and harnessing adversarial examples[C]//Proceedings of the 3rd International Conference on Learning Representations（ICLR），2015.
[7] MIWA M，BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg，PA：ACL Press，2016：1105-1116.
[8] LI F，ZHANG M S，FU G H，et al.A neural joint model for entity and relation extraction from biomedical text[J].BMC Bioinformatics，2017，18（1）：198-208.
[9] KATIYAR A，CARDIE C.Going out on a limb：joint extraction of entity mentions and relations without dependency trees[M].Stroudsburg：Association for Computational Linguistics，2017：917-928.
[10] 孙长志.基于深度学习的联合实体关系抽取[D].上海：华东师范大学，2020.
SUN C Z.Joint entity relationship extraction based on deep learning[D].Shanghai：East China Normal University，2020.
[11] ZHENG S，WANG F，BAO H，et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg，PA：ACL，2017：1227-1236.
[12] BEKOULIS G，DELEU J，DEMEESTER T，et al.Joint entity recognition and relation extraction as a multi-head selection problem[J].Expert Systems with Application，2018，114：34-45.
[13] BEKOULIS G，DELEU J，DEMEESTER T，et al.Adversarial training for multicontext joint entity and relation extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels，Belgium：ACL Press，2018：2830-2836.
[14] ZHANG Y，QI P，MANNING C.Graph convolution over pruned dependency trees improves relation extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels，Belgium：ACM，2018：2205-2215.
[15] ZHU H，LIN Y K，LIU Z Y，et al.Graph neural networks with generated parameters for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence，Italy：ACM，2019：1331-1339.
[16] FU T J，MA W Y.GraphRel：modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg：ACL Press，2019：1409-1418.
[17] SRIVASTAVA N，HINTON G，KRIZHEVSKY A，et al.Dropout：a simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research，2014，15（1）：1929-1958.
[18] MIYATO T，DAI A M，GOODFELLOW I.Adversarial training methods for semisupervised text classification[C]//Proceedings of the International Conference on Learning Representations，Toulon，France，2017.
[19] WU Y，BAMMAN D，RUSSELL S.Adversarial training relation extraction[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing（EMNLP 2017）.Stroudsburg，PA：ACL，2017：1778-1783.
[20] YASUNAGA M，KASAI J，RADEV D.Robust multilingual part-of-speech tagging via adversarial training[C]//Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，New Orleans，USA，2018.
[21] ZENG X，ZENG D，HE S，et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.Stroudsburg：ACL Press，2018：506-514.
[22] WANG Q，LV L，YU B，et al.End-to-end relation extraction using graph convolutional network with a novel entity attention[C]//2020 IEEE 6th International Conference on Computer and Communications，2020.
[23] ZGA B，YZA B，YHA B.Joint entity and relation extraction model based on rich semantics[J].Neurocomputing，2021，429：132-140.