Joint Extraction of Entities and Relations Model for Single-Step Span-Labeling

doi:10.3778/j.issn.1002-8331.2112-0418

Abstract

Abstract: As an upstream task in many fields such as knowledge graph, relation extraction has a wide range of application value and has received extensive attention in recent years. At present, the problem of exposure bias is common in relation extraction models, and the problems of entity nesting and entity overlapping are common in extracted text, which seriously affect the performance of the model. Therefore, this paper proposes an entity-relationship extraction model（span-labeling based model, SLM） based on Span labeling, which mainly includes：transforming entity-relation extraction problem into span labeling problem; the tokens are combined and arranged and re-tiled into a Span sequence. LSTM and multi-head self-attention mechanism are used to extract deep semantic features of the span. An entity relation label is designed, and a multi-layer labeling method is used for relation label classification. Experiments are carried out on the English datasets NYT and WebNLG. Compared with the baseline model, the F1 value is significantly improved, which verifies the effectiveness of the model, indicating that the model can effectively solve the above problems.

Key words: relation extraction, joint extraction, span-labeling, mapping strategy, exposure bias, entity nesting, entity overlap

摘要： 关系抽取作为知识图谱等诸多领域的上游任务，具有广泛应用价值，近年来受到广泛关注。关系抽取模型普遍存在暴露偏差问题，抽取文本普遍存在实体嵌套和实体重叠问题，这些问题严重影响了模型性能。因此，提出了一种基于片段标注的实体关系联合抽取模型（span-labeling based model，SLM），主要包括：将实体关系抽取问题转化为片段标注问题；使用滑动窗口和三种映射策略将词元（token）序列进行组合排列重新平铺成片段（span）序列；使用LSTM和多头自注意力机制进行片段深层语义特征提取；设计了实体关系标签，使用多层标注方法进行关系标签分类。在英文数据集NYT、WebNLG上进行实验，相对于基线模型F1值显著提高，验证了模型的有效性，能有效解决上述问题。

关键词: 关系抽取, 联合抽取, 片段标注, 映射策略, 暴露偏差, 实体嵌套, 实体重叠

ZHENG Zhaoqian, HAN Dongchen, ZHAO Hui. Joint Extraction of Entities and Relations Model for Single-Step Span-Labeling[J]. Computer Engineering and Applications, 2023, 59(9): 130-139.

郑肇谦, 韩东辰, 赵辉. 单步片段标注的实体关系联合抽取模型[J]. 计算机工程与应用, 2023, 59(9): 130-139.

References

[1] 李冬梅，张扬，李东远，等.实体关系抽取方法研究综述[J].计算机研究与发展，2020，57（7）：1424-1448.
LI D M，ZHANG Y，LI D Y，et al.Review of entity relation extraction methods[J].Journal of Computer Research and Development，2020，57（7）：1424-1448.
[2] 鄂海红，张文静，肖思琪，等.深度学习实体关系抽取研究综述[J].软件学报，2019，30（6）：1793-1818.
E H H，ZHANG W J，XIAO S Q，et al.Survey of entity relationship extraction based on deep learning[J].Journal of Software，2019，30（6）：1793-1818.
[3] 冯钧，张涛，杭婷婷.重叠实体关系抽取综述[J].计算机工程与应用，2022，58（1）：1-11.
FENG J，ZHANG T，HANG T T.Survey of overlapping entities and relations extraction[J].Computer Engineering and Applications，2022，58（1）：1-11.
[4] SOCHER R，HUVAL B，MANNING C D，et al.Semantic compositionality through recursive matrix-vector spaces[C]//Joint Conference on Empirical Methods in Natural Language Processing & Computational Natural Language Learning，2012：1201-1211.
[5] ZENG D，LIU K，LAI S，et al.Relation classification via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics：Technical Papers，2014：2335-2344.
[6] NGUYEN T H，GRISHMAN R.Relation extraction：perspective from convolutional neural networks[C]//Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing，2015：39-48.
[7] 吴天昊，古丽拉·阿东别克.基于神经元块级别注意力机制的LSTM关系抽取[J].计算机应用研究，2020，37（S2）：76-79.
WU T H，GULILA·ALTENBEK.LSTM relation extraction based on neuron block level attention mechanism[J].Application Research of Computers，2020，37（S2）：76-79.
[8] XU K，FENG Y，HUANG S，et al.Semantic relation classification via convolutional neural networks with simple negative sampling[J].arXiv：1506.07650，2015.
[9] ZHANG S，ZHENG D，HU X，et al.Bidirectional long short-term memory networks for relation classificationC]//Proceedings of the 29th Pacific Asia Conference on Language，Information and Computation，2015：73-78.
[10] ZHONG Z，CHEN D.A frustratingly easy approach for joint entity and relation extraction[J].arXiv：2010.12812，2020.
[11] 赵敏钧，赵亚伟，赵雅捷，等.一种新的基于深度学习的重叠关系联合抽取模型[J].中国科学院大学学报，2022，39（2）：240-251.
ZHAO M J，ZHAO Y W，ZHAO Y J，et al.A new joint model for extracting overlapping relations based on deep learning[J].Journal of University of Chinese Academy of Sciences，2020，39（2）：240-251.
[12] GUPTA P，SCHIITZE H，ANDRASSY B.Table filling multi-task recurrent neural network for joint entity and relation extraction[C]//Proceedings of the 26th International Conference on Computational Linguistics：Technical Papers，2016：2537-2547.
[13] ZHENG S，WANG F，BAO H，et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2017：1227-1236.
[14] DAI D，XIAO X，LYU Y，et al.Joint extraction of entities and overlapping relations using position-attentive sequence labeling[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：6300-6308.
[15] ZENG D，ZHANG H，LIU Q.CopyMTL：copy mechanism for joint extraction of entities and relations with multi-task learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：9507-9514.
[16] ZHANG R H，LIU Q，FAN A X，et al.Minimize exposure bias of Seq2Seq models in joint entity and relation extraction[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing：Findings，2020：236-246.
[17] WEI Z，SU J，WANG Y，et al.A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：1476-1488.
[18] MIWA M，BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2016：1105-1116.
[19] BEKOULIS G，DELEU J，DEMEESTER T，et al.Joint entity recognition and relation extraction as a multi-head selection problem[J].Expert Systems with Applications，2018，114：34-45.
[20] KATIYAR A，CARDIE C.Going out on a limb：joint extraction of entity mentions and relations without dependency trees[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2017：917-928.
[21] LI X，YIN F，SUN Z，et al.Entity-relation extraction as multi-turn question answering[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：1340-1350.
[22] DIXIT K，AL-ONAIZAN Y.Span-level model for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：5308-5314.
[23] EBERTS M，ULGES A.Span-based joint entity and relation extraction with transformer pre-training[J].arXiv：1909.07755，2019.
[24] FU T J，LI P H，MA W Y.GraphRel：modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：1409-1418.
[25] WANG Y，YU B，ZHANG Y，et al.TPLinker：single-stage joint extraction of entities and relations through token pair linking[C]//Proceedings of the 28th International Conference on Computational Linguistics，2020：1572-1582.
[26] ZHENG H，WEN R，CHEN X，et al.PRGC：potential relation and global correspondence based joint relational triple extraction[J].arXiv：2106.09895，2021.
[27] SUI D，CHEN Y，LIU K，et al.Joint entity and relation extraction with set prediction networks[J].arXiv：2011. 01675，2020.
[28] DEVLIN J，CHANG M W，LEE K，et al.Bert：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[29] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural computation，1997，9（8）：1735-1780.
[30] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017：5998-6008.
[31] RIEDEL S，YAO L，MCCALLUM A.Modeling relations and their mentions without labeled text[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Berlin，Heidelberg：Springer，2010：148-163.
[32] MORYOSSEF A，GOLDBERG Y，DAGAN I.Step-by-step：separating planning from realization in neural data-to-text generation[J].arXiv：1904.03396，2019.
[33] ZENG X，ZENG D，HE S，et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2018：506-514.
[34] YU B，ZHANG Z，SHU X，et al.Joint extraction of entities and relations based on a novel decomposition strategy[J].arXiv：1909.04273，2019.
[35] NAYAK T，NG H T.Effective modeling of encoder-decoder architecture for joint entity and relation extraction[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：8528-8535.