Survey of Overlapping Entities and Relations Extraction

doi:10.3778/j.issn.1002-8331.2107-0359

Abstract

Abstract: Entity relation extraction can extract knowledge of facts from text, which is an important task in the field of natural language processing. Traditional relation extraction pays more attention to the relation of a single entity pair, However, there are more than one entity pair as well as overlaps between entities. Therefore, it is important to research overlapping entity relation extraction. With the development of the task, there are three classes method：Seq2Seq, graph-based and pre-trained language model. Seq2Seq-based methods make use of labeling strategies and copy mechanisms, graph-based methods are mainly based on static graphs and dynamic graphs, and pre-training language models are methods mining potential semantic features on BERT. This paper reviews the development of the task, discusses and analyzes the pros sand cons of each model. It combines with the recent developments of research to prospects for future research directions.

Key words: overlapping entities and relations, deep learning, graph neural network, pre-training language model

摘要： 实体关系抽取能够从文本中提取事实知识，是自然语言处理领域中重要的任务。传统关系抽取更加关注于单实体对的关系，但是句子内包含不止一对实体且实体间存在重叠现象，因此重叠实体关系抽取任务具有重大研究价值。任务发展至今，总体可以分为基于序列到序列、基于图和基于预训练语言模型三种方式。基于序列到序列的方式主要以标注策略和复制机制的方法为主，基于图的方式主要以静态图和动态图的方法为主，基于预训练语言模型的方式主要以BERT挖掘潜在语义特征的方法为主。回顾该任务的发展历程，讨论分析每种模型的优势及不足点；结合目前研究的最近动态，对未来的研究方向进行展望。

关键词: 重叠实体关系抽取, 深度学习, 图神经网络, 预训练语言模型

FENG Jun, ZHANG Tao, HANG Tingting. Survey of Overlapping Entities and Relations Extraction[J]. Computer Engineering and Applications, 2022, 58(1): 1-11.

冯钧, 张涛, 杭婷婷. 重叠实体关系抽取综述[J]. 计算机工程与应用, 2022, 58(1): 1-11.

References

[1] 鄂海红，张文静，肖思琪，等.深度学习实体关系抽取研究综述[J].软件学报，2019，30（6）：1793-1818.
E H H，ZHANG W J，XIAO S Q，et al.Survey of entity relationship extraction based on deep learning[J].Journal of Software，2019，30（6）：1793-1818.
[2] 李冬梅，张扬，李东远，等.实体关系抽取方法研究综述[J].计算机研究与发展，2020，57（7）：1424-1448.
LI D M，ZHANG Y，LI D Y，et al.Review of entity relation extraction methods[J].Journal of Computer Research and Development，2020，57（7）：1424-1448.
[3] KAMBHATLA N.Combining lexical，syntactic，and semantic features with maximum entropy models for information extraction[C]//Proceedings of the ACL Interactive Poster and Demonstration Sessions，2004：178-181.
[4] ZELENKO D，AONE C，RICHARDELLA A.Kernel methods for relation extraction[J].Journal of Machine Learning Research，2003，3：1083-1106.
[5] CHAPELLE O，ZIEN A.Semi-supervised learning[M].Cambridge：MIT Press，2006：3-14.
[6] ZHU J，NIE Z，LIU X，et al.StatSnowball：A statistical approach to extracting entity relationships[C]//Proceedings of the 18th International Conference on World Wide Web，2009：101-110.
[7] ZHANG Z.Weakly-supervised relation classification for information extraction[C]//Proceedings of the 13th ACM International Conference on Information and Knowledge Management，2004：581-588.
[8] HOFFMANN R，ZHANG C，LING X，et al.Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics：Human Language Technologies，2011：541-550.
[9] 秦兵，刘安安，刘挺.无指导的中文开放式实体关系抽取[J].计算机研究与发展，2015，52（5）：1029-1035.
QIN B，LIU A A，LIU T.Unsupervised Chinese open entity relation extraction[J].Journal of Computer Research and Development，2015，52（5）：1029-1035.
[10] HINTON G E，SALAKHUTDINOV R R.Reducing the dimensionality of data with neural networks[J].Science，2006，313：504-507.
[11] BAI F，RITTER A.Structured minimally supervised learning for neural relation extraction[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2019：3057-3069.
[12] ZHENG S，HAO Y，LU D，et al.Joint entity and relation extraction based on a hybrid neural network[J].Neurocomputing，2017，257：1083-1106.
[13] MINTZ M，BILLS S，SNOW R，et al.Distant supervision for relation extraction without labeled data[C]//Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP，2009：1003-1011.
[14] SUTSKEVER I，VINYALS O，LE Q V.Sequence to sequence learning with neural networks[C]//Proceedings of Annual Conference on Neural Information Processing Systems，2014：3104-3112.
[15] ZHENG S，WANG F，BAO H，et al.Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics，2017：1227-1236.
[16] YU B W，ZHANG Z Y，SHU X B，et al.Joint extraction of entities and relations based on a novel decomposition strategy[C]//Proceedings of the 24th European Conference on Artificial Intelligence，2020：2282-2289.
[17] 田佳来，吕学强，游新冬，等.基于分层序列标注的实体关系联合抽取方法[J].北京大学学报（自然科学版），2021，57（1）：53-60.
TIAN J L，LYU X Q，YOU X D，et al.Joint extraction of entities and relations based on hierarchical sequence labeling[J].Acta Scientiarum Naturalium Universitatis Pekinensis，2021，57（1）：53-60.
[18] 赵敏钧，赵亚伟，赵雅捷，等.一种新的基于深度学习的重叠关系联合抽取模型[J].中国科学院大学学报，DOI：10.7523/j.ucas.2020.0026.
ZHAO M J，ZHAO Y W，ZHAO Y J，et al.A new joint model for extracting overlapping relations based on deep learning[J].Journal of University of Chinese Academy of Sciences，DOI：10.7523/j.ucas.2020.0026.
[19] LIN K，MIAO K，HONG W，et al.Relation extraction based on relation label constraints[C]//Proceedings of 2020 IEEE 6th International Conference on Computer and Communications（ICCC），2020：2166-2170.
[20] ZHUANG C，ZHANG N，JIN X，et al.Joint extraction of triple knowledge based on relation priority[C]//2020 IEEE International Conference on Parallel & Distributed Processing with Applications，Big Data & Cloud Computing，Sustainable Computing & Communications，Social Computing & Networking，2020：562-569.
[21] YUAN Y，X ZHOU，PAN S，et al.A relation-specific attention network for joint entity and relation extraction[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence，2020：4054-4060.
[22] LIU J，CHEN S，WANG B，et al.Attention as relation：Learning supervised multi-head self-attention for relation extraction[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence，2020：3787-3793.
[23] LUO X，LIU W，MA M，et al.BiTT：Bidirectional tree tagging for joint extraction of overlapping entities and relations[J].arXiv：2008.13339，2020.
[24] MA L，REN H，ZHANG X.Effective cascade dual-decoder model for joint entity and relation extraction[J].arXiv：2106.14163，2021.
[25] WEI Z，SU J，WANG Y，et al.A novel cascade binary tagging framework for relational triple extraction[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：1476-1488.
[26] WANG L，XIONG C，DENG N.A research on overlapping relationship extraction based on multi-objective dependency[C]//Proceedings of the 15th International Conference on Computer Science & Education，2020：618-622.
[27] WANG Y，YU B，ZHANG Y，et al.TPLinker：Single-stage joint extraction of entities and relations through token pair linking[C]//Proceedings of the 28th International Conference on Computational Linguistics，2020：1572-1582.
[28] ZENG X，ZENG D，HE S，et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，2018：506-514.
[29] ZENG X，HE S，ZENG D，et al.Learning the extraction order of multiple relational facts in a sentence with reinforcement learning[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，2019：367-377.
[30] BAI C，PAN L，LUO S，et al.Joint extraction of entities and relations by a novel end-to-end model with a double-pointer module[J].Neurocomputing，2020，377：325-333.
[31] ZENG D，ZHANG H，LIU Q.Copymtl：Copy mechanism for joint extraction of entities and relations with multi-task learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：9507-9514.
[32] WANG X，LI Q，DING X，et al.A new method for complex triplet extraction of biomedical texts[C]//Proceedings of the 12th International Conference on Knowledge Science，Engineering and Management，2019：146-158.
[33] NAYAK T，NG H T.Effective modeling of encoder-decoder architecture for joint entity and relation extraction[C]//Proceedings of the 34th Conference on Artificial Intelligence，2020：8528-8535.
[34] WANG S，ZHANG Y，CHE W，et al.Joint extraction of entities and relations based on a novel graph scheme[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence，2018：4461-4467.
[35] HONG Y，LIU Y，YANG S，et al.Improving graph convolutional networks based on relation-aware attention for end-to-end relation extraction[J].IEEE Access，2020，8：51315-51323.
[36] WANG X，WANG D，JI F.A span-based model for joint entity and relation extraction with relational graphs[C]//Proceedings of 2020 IEEE International Conference on Parallel & Distributed Processing with Applications，Big Data & Cloud Computing，Sustainable Computing & Communications，Social Computing & Networking，2020：513-520.
[37] FU T J，LI P H，MA W Y.GraphRel：Modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：1409-1418.
[38] FEI H，REN Y，JI D.Boundaries and edges rethinking：An end-to-end neural model for overlapping entity relation extraction[J].Information Processing & Management，2020，57（6）：102311.
[39] DUAN G，MIAO J，HUANG T，et al.A relational adaptive neural model for joint entity and relation extraction[J].Frontiers in Neurorobotics，2021，15：635492.
[40] DEVLIN J，CHANG M W，LEE K，et al.BERT：Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2019：4171-4186.
[41] LI C，TIAN Y.Downstream model design of pre-trained language model for relation extraction task[J].arXiv：2004.03786，2020.
[42] SUI D，CHEN Y，LIU K，et al.Joint entity and relation extraction with set prediction networks[J].arXiv：2011. 01675，2020.
[43] LIU L，WANG M，HE X，et al.Extracting relational facts based on hybrid Syntax-Guided transformer and pointer network[J].Journal of Intelligent & Fuzzy Systems，2021，40（6）：12167-12183.
[44] YE H，ZHANG N，DENG S，et al.Contrastive triple extraction with generative transformer[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence，2021：14257-14265.
[45] HANG T，FENG J，WU Y，et al.Joint extraction of entities and overlapping relations using source-target entity labeling[J].Expert Systems with Applications，2021，177：114853.
[46] FENG J，HUANG M，ZHAO L，et al.Reinforcement learning for relation classification from noisy data[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence，2018：5779-5786.
[47] TAKANOBU R，ZHANG T，LIU J，et al.A hierarchical framework for relation extraction with reinforcement learning[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence，2019：7072-7079.
[48] WANG X，JI H，SHI C，et al.Heterogeneous graph attention network[C]//Proceedings of World Wide Web Conference，2019：2022-2032.
[49] GAO H，PEI J，HUANG H.ProGAN：Network embedding via proximity generative adversarial network[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining，2019：1308-1316.
[50] ZHAO K，XU H，CHENG Y，et al.Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction[J].Knowledge-Based Systems，2021，219：106888.
[51] SUN C，QIU X，XU Y，et al.How to fine-tune bert for text classification?[C]//Proceedings of China National Conference on Chinese Computational Linguistics，2019：194-206.
[52] LIU W，ZHOU P，ZHAO Z，et al.K-bert：Enabling language representation with knowledge graph[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：2901-2908.
[53] LIU Y，OTT M，GOYAL N，et al.Roberta：A robustly optimized BERT pretraining approach[J].arXiv：1907. 11692，2019.
[54] CHEN Y，ZHANG Y，HU C，et al.Jointly extracting explicit and implicit relational triples with reasoning pattern enhanced binary pointer network[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2021：5694-5703.