融合交互注意力网络的实体和关系联合抽取模型

doi:10.3778/j.issn.1002-8331.2301-0154

摘要/Abstract

摘要： 实体关系三元组的抽取效果直接影响后期知识图谱构建的质量，而传统流水线式和联合式抽取的模型，并没有对句子级别和关系级别的语义特征进行有效建模，从而导致模型性能的缺失。为此，提出一种融合句子级别和关系级别的交互注意力网络的实体和关系联合抽取模型RSIAN，该模型通过交互注意力网络来学习句子级别和关系级别的高阶语义关联，增强句子和关系之间的交互，辅助模型进行抽取决策。在构建的中文旅游数据集（TDDS）的Precision、Recall和F1值分别为0.872、0.760和0.812，其性能均优于其他对比模型；为了进一步验证该模型在英文联合抽取上的性能，在公开英文数据集NYT和Webnlg上进行实验，该模型的F1值相比基线模型RSAN模型分别提高了0.014和0.013，并且该模型在重叠三元组的分析实验也均取得了优于基线模型的性能且更稳定。

关键词: 交互注意力网络, 句子级别, 关系级别, 实体和关系联合抽取, 注意力机制, 重叠三元组

Abstract: Entity relationship triples extraction effect has a direct impact on the construction of knowledge graphs in the later stage. The traditional pipeline and joint extraction models do not effectively model the semantic features at sentence level and relationship level, which leads to the lack of model performance. To this end, a joint entity and relation extraction model RSIAN that fuses the semantic features at the sentence level and relation level is proposed, which learns the higher-order semantic associations at the sentence level and relation level through an interactive attention network to enhance the interaction between sentences and relations and assist the model in extraction decisions. The precision, recall, and F1 values of the Chinese tourism dataset (TDDS) constructs in this paper are 0.872, 0.760, and 0.812, respectively, all of which outperform the current mainstream model. To further validate the performance of the model on joint extraction in English, experiments are conducted on the publicly available English datasets NYT and Webnlg. The F1 values of the model compared to the baseline RSAN model are increased by 0.014 and 0.013, respectively, and this model also achieves better performance than the baseline model in the analysis experiments of overlapping triads.

Key words: interactive attention network, sentence-level, relationship-level, joint extraction model of entity and relation, attention mechanism, overlapping triple

郝小芳, 张超群, 李晓翔, 王大睿. 融合交互注意力网络的实体和关系联合抽取模型[J]. 计算机工程与应用, 2024, 60(8): 156-164.

HAO Xiaofang, ZHANG Chaoqun, LI Xiaoxiang, WANG Darui. Joint Entity Relation Extraction Model Based on Interactive Attention[J]. Computer Engineering and Applications, 2024, 60(8): 156-164.

参考文献

[1] YEE S C, DAN R. Exploiting syntactico-semantic structures for relation extraction[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 2011: 551-560.
[2] DIEDERIK P K, JIMMY B. Adam: a method for stochastic optimization[J]. arXiv:1412.6980, 2014.
[3] LI H, WU X, LI Z, et al. A relation extraction method of Chinese named entities based on location and semantic features[J]. Applied Intelligence, 2013, 38(1): 1-15.
[4] MAKOTO M, YUTAKA S. Modeling joint entity and relation extraction with table representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1858-1869.
[5] ZHOU P, ZHENG S, XU J, et al. Joint extraction of multiple relations and entities by using a hybrid neural network[C]//Proceedings of the Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, 2017: 135-146.
[6] QIAO B, ZOU Z, HUANG Y, et al. A joint model for entity and relation extraction based on BERT[J]. Neural Computing and Applications, 2022, 34(5): 3471-3481.
[7] 肖立中, 臧中兴, 宋赛赛. 融合自注意力的关系抽取级联标记框架研究[J].计算机工程与应用, 2023, 59(3): 77-83.
XIAO L Z, ZANG Z X, SONG S S, Research on cascaded labeling framework for relation extraction with self-attention[J]. Computer Engineering and Applications, 2023, 59(3): 77-83.
[8] 雷景生, 剌凯俊, 杨胜英, 等. 基于上下文语义增强的实体关系联合抽取[J].计算机应用, 2023, 43(5): 1438-1444.
LEI J S, CI K J, YANG S Y, et al, Joint entity and relation extraction based on context semantic enhancement[J]. Journal of Computer Applications, 2023, 43(5): 1438-1444.
[9] 何松泽, 王婷, 梁佳莹, 等.基于自注意力机制模拟实体信息的实体关系抽取[J].计算机系统应用, 2023, 32(2): 364-370.
HE S Z, WANG T, LIANG J Y, et al. Entity relation extraction simulation of entity information based on self-attention mechanism[J]. Computer Systems & Applications, 2023, 32(2): 364-370.
[10] ZENG X R, ZENG D J, HE S Z, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2018: 506-514.
[11] YUAN Y, ZHOU X, PAN S, et al. A relation-specific attention network for joint entity and relation extraction[C]//Proceedings of the International Joint Conference on Artificial Intelligence, 2020: 4054-4060.
[12] LAI T, CHENG L, WANG D, et al. RMAN: relational multi-head attention neural network for joint extraction of entities and relations[J]. Applied Intelligence, 2022, 52(3): 3132-3142.
[13] ZELENKO D, AONE C, RICHARDELLA A. Kernel methods for relation extraction[J]. Journal of Machine Learning Research, 2003, 3: 1083-1106.
[14] MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009: 1003-1011.
[15] GORMLEY M R, YU M, DREDZE M. Improved relation extraction with feature-rich compositional embedding models[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2015: 1774-1784.
[16] ZHENG Z, CHEN D. A frustratingly easy approach for entity and relation extraction[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021: 50-61.
[17] MIWA M, SASAKI Y. Modeling joint entity and relation extraction with table representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014: 1858-1869.
[18] REN X, WU Z, HE W, et al. CoType: joint extraction of typed entities and relations with knowledge bases[C]//Proceedings of the International Conference on World Wide Web, 2017: 1015-1024.
[19] MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2016: 1105-1116.
[20] KATIYAR A, CARDIE C. Going out on a limb: joint extraction of entity mentions and relations without dependency trees[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2017: 917-928.
[21] ZHENG S, WANG F, BAO H, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2017: 1227-1236.
[22] FU T J, LI P H, MA W Y. GraphRel: modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 1409-1418.
[23] HONG Y, LIU Y, YANG S, et al. Improving graph convolutional networks based on relation-aware attention for end-to-end relation extraction[J]. IEEE Access, 2020, 8: 51315-51323.
[24] TAKANOBU R, ZHANG T, LIU J, et al. A hierarchical framework for relation extraction with reinforcement learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2019: 7072-7079.
[25] FEI H, REN Y, JI D. Boundaries and edges rethinking: an end-to-end neural model for overlapping entity relation extraction[J]. Information Processing & Management, 2020, 57(6): 102311.
[26] NAYAK T, NG H T. Effective modeling of encoder-decoder architecture for joint entity and relation extraction[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 8528-8535.
[27] YU B, ZHANG Z, SHU X, et al. Joint extraction of entities and relations based on a novel decomposition strategy[C]//Proceedings of the European Conference on Artificial Intelligence, 2020: 2282-2289.
[28] WEI M, XU Z, HU J. Entity relationship extraction based on BiLSTM and Attention mechanism[C]//Proceedings of the International Conference on Artificial Intelligence and Information Systems, 2021: 1-5.
[29] 常思杰, 林浩田, 江静.融合双阶段解码的实体关系联合抽取方法[J].计算机工程与应用, 2023, 59(20): 138-146.
CHANG S J, LIN H T, JIANG J. Joint entity relation extraction based on two-stage decoding[J].Computer Engineering and Applications, 2023, 59(20): 138-146.
[30] 陈赟, 古丽拉·阿东别克, 马雅静.旅游领域实体和关系联合抽取方法研究[J].计算机工程与应用, 2022, 58(18): 284-296.
CHEN Y, GULILA A, MA Y J. Research on joint extraction method of entity and relation in tourism domain[J]. Computer Engineering and Applications, 2022, 58(18): 284-296.
[31] WEI Z, SU J, WANG Y, et al. A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2020: 1476-1488.
[32] CLAIRE G, ANASTASIA S, SHASHI N, et al. Creating training corpora for NLG micro-planners[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics, 2017: 179-188.