Joint Entity Relation Extraction Based on Two-Stage Decoding

doi:10.3778/j.issn.1002-8331.2206-0459

Abstract

Abstract: The concatenated decoding method has solved some overlapping problems by directly optimizing the triples in the existing joint entity relation extraction task. However, this method will cause the imbalance problem of entity recognition to the decoded entities under a specific relation. Although the order problem of triples has been solved only by use of the set prediction method to synchronous decode entities and relations, it also leads to the problems of weak connection among different entities and poor interaction between entities and relations. This paper proposes a joint extraction model of entity relations that integrates two-stage decoding in order to further improve the effect of the joint extraction model, which including entity decoding in the cascade decoding stage and relationship decoding in the set prediction network stage. The model in this paper is divided into three parts：Firstly, Bert is used for encoding, which pays attention to the context information effectively. Secondly, the recognition of entities uses a cascade decoding strategy, which can extract the entity information without relationship constraints, and adequate identification of entities. Finally, the triples fused with entity information are embedded into the ensemble prediction network to decode the entity-relation triplet and to strengthen the connection between entity and relation. Based on the open data sets of the New York Times（NYT）, WebNLG and ACE2005, the experimental results show that the model proposed in this paper is basically superior to the baseline model, which verifies the effectiveness of the model.

Key words: entity relation joint extraction, overlapping problems, cascade decoding, ensemble prediction

摘要： 在现有的实体关系联合抽取任务中，级联解码的方法直接对三元组进行优化，解决了一部分重叠问题，但是在特定关系下解码的实体，造成实体识别不平衡问题。仅用集合预测的方法可以同时解码出实体和关系，虽然解决了三元组的顺序问题，但也导致实体之间联系性不强、实体和关系之间交互性差的问题。为了进一步提高联合抽取模型的效果，提出一种融合双阶段解码的实体关系联合抽取模型，包括级联策略下的实体解码与集合预测网络阶段的关系解码。该模型分为三个部分：采用Bert进行编码，有效关注到了上下文的信息；采用级联解码的策略先对实体识别，得到不受关系限制的实体信息，充分识别实体；将融合了实体信息的表示嵌入集合预测网络解码出实体-关系三元组，加强实体与关系的联系。在公开数据集纽约时报（The New York Times，NYT）、WebNLG和ACE2005上的实验结果表明，所提出的模型基本优于基线模型，验证了该模型的有效性。

关键词: 实体关系联合抽取, 重叠问题, 级联解码, 集合预测

CHANG Sijie, LIN Haotian, JIANG Jing. Joint Entity Relation Extraction Based on Two-Stage Decoding[J]. Computer Engineering and Applications, 2023, 59(20): 138-146.

常思杰, 林浩田, 江静. 融合双阶段解码的实体关系联合抽取方法[J]. 计算机工程与应用, 2023, 59(20): 138-146.

References

[1] BAI F，RITTER A.Structured minimally supervised learning for neural relation extraction[J].arXiv：1904.00118，2019.
[2] ZHENG S，HAO Y，LU D，et al.Joint entity and relation extraction based on a hybrid neural network[J].Neurocomputing，2017，257：59-66.
[3] ZENG D，LIU K，LAI S，et al.Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014，the 25th International Conference on Computational Linguistics：Technical Papers，2014：2335-2344.
[4] SOCHER R，HUVAL B，MANNING C D，et al.Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning，2012：1201-1211.
[5] MIWA M，BANSAL M.End-to-end relation extraction using lstms on sequences and tree structures[J].arXiv：1601.00770，2016.
[6] ZHENG S，WANG F，BAO H，et al.Joint extraction of entities and relations based on a novel tagging scheme[J].arXiv：1706.05075，2017.
[7] 冯钧，张涛，杭婷婷.重叠实体关系抽取综述[J].计算机工程与应用，2022，58（1）：1-11.
FENG J，ZHANG T，HANG T T.Survey of overlapping entities and relations extraction[J].Computer Engineering and Applications，2022，58（1）：1-11.
[8] WEI Z，SU J，WANG Y，et al.A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020.
[9] SUI D，CHEN Y，LIU K，et al.Joint entity and relation extraction with set prediction networks[J].arXiv：2011.01675，2020.
[10] DEVLIN J，CHANG M W，LEE K，et al.Bert：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[11] LI J，SUN A，HAN J，et al.A survey on deep learning for named entity recognition[J].IEEE Transactions on Knowledge and Data Engineering，2020：2981314.
[12] ZHANG Q，CHEN M，LIU L.A review on entity relation extraction[C]//2017 Second International Conference on Mechanical，Control and Computer Engineering（ICMCCE），2017：178-183.
[13] WANG L，ZHU C，MELO G D，et al.Relation classification via multi-level attention CNNs[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics，2016：1298-1307.
[14] CAI R，ZHANG X D，WANG H F.Bidirectional recurrent convolutional neural network for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics，2016：756-765.
[15] ZHONG Z，CHEN D.A frustratingly easy approach for entity and relation extraction[C]//North American Chapter of the Association for Computational Linguistics，2021.
[16] LI F，ZHANG M，FU G，et al.A neural joint model for extracting bacteria and their locations[C]//Pacific-Asia Conference on Knowledge Discovery and Data Mining.Cham：Springer，2017：15-26.
[17] KATIYAR A，CARDIE C.Going out on a limb：joint extraction of entity mentions and relations without dependency trees[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics，2017：917-928.
[18] WANG J，LU W.Two are better than one：joint entity and relation extraction with table-sequence encoders[J].arXiv：2010.03851，2020.
[19] LI X，YIN F，SUN Z，et al.Entity-relation extraction as multi-turn question answering[J].arXiv：1905.05529，2019.
[20] MA L，REN H，ZHANG X.Effective cascade dual-decoder model for joint entity and relation extraction[J].arXiv：2106.14163，2021.
[21] DIXIT K，AL-ONAIZAN Y.Span-level model for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：5308-5314.
[22] 田佳来，吕学强，游新冬，等.基于分层序列标注的实体关系联合抽取方法[J].北京大学学报（自然科学版），2021，57（1）：53-60.
TIAN J L，LV X Q，YOU X D，et al.A joint extraction method of entity relations based on hierarchical sequence annotation[J].Journal of Peking University（Natural Science Edition），2021，57（1）：53-60.
[23] ZENG X，ZENG D，HE S，et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，2018：506-514.
[24] ZENG D，ZHANG H，LIU Q.CopyMTL：copy mechanism for joint extraction of entities and relations with multi-task learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：9507-9514.
[25] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017.
[26] 彭晏飞，王瑞华，张睿思.基于双集合预测网络的实体关系联合抽取模型[J].计算机科学与探索，2023，17（7）：1690-1699.
PENG Y F，WANG R H，ZHANG R S.Dual set prediction networks based joint extraction of entity and relation[J].Journal of Frontiers of Computer Science & Technology，2023，17（7）：1690-1699.
[27] FU T J，LI P H，MA W Y.GraphRel：modeling text as relational graphs for joint entity and relation extraction[C]//Meeting of the Association for Computational Linguistics，2019.