Fast Model for Joint Extraction of Entity and Relation

doi:10.3778/j.issn.1002-8331.2204-0327

Abstract

Abstract: Extracting entities and relations from plain text is a key technique for knowledge and question answering tasks. The traditional multi-head model predicts the relation type of all segment pairs, while the number of negative labels for segment pairs is much larger than positive labels due to the sparsity of relations. At the same time, this calculation method causes the calculation amount to be proportional to the square of sentence length, which reduces the practicability of the model. To solve this problem, a fast extraction model of entity and relation is proposed. For the named entity recognition task, the start and end labels of entities are predicted by two pointer networks, respectively. Semantic segment pairs that do not contain entity end tags are removed in the relation extraction task. This method reduces the number of segment pairs and speeds up inference for relation extraction tasks. To demonstrate the effectiveness of the model, experiments are conducted on the English news dataset ACE05 and the Dutch real estate dataset DREC. The experimental results show that the model achieves competitive performance compared with the baseline model, and its inference speed is improved by 1.4 times on ACE05 and 2.1 times on DREC.

Key words: entity recognition, relation extraction, neural network, natural language processing, information extraction

摘要： 从纯文本中抽取实体和关系是知识和问答任务的关键技术。传统的多头模型预测所有片段对的关系类型，而由于关系的稀疏性，片段对的负标签数量远大于正标签。同时，该计算方式导致计算量与句长度的二次方成正比，降低了模型的实用性。为解决该问题，快速实体关系抽取模型被提出。对于命名实体识别任务，实体的开始和结束标签分别对两个指针网络预测。在关系抽取任务中删除了不包含实体结束标签的语义片段对。该方法减少了片段对的数量并加快了关系抽取任务的推理速度。为了证明模型的有效性，在英语新闻数据集ACE05和荷兰语房地产数据集DREC上进行了实验。实验结果表明，与基线模型相比，该模型取得了有竞争力的性能，其推理速度在ACE05上提高了约1.4倍，在DREC上提高了约2.1倍。

关键词: 实体识别, 关系抽取, 神经网络, 自然语言处理, 信息抽取

YANG Dong, TIAN Shengwei, YU Long, ZHOU Tiejun, WANG Bo. Fast Model for Joint Extraction of Entity and Relation[J]. Computer Engineering and Applications, 2023, 59(13): 164-170.

杨冬, 田生伟, 禹龙, 周铁军, 王博. 快速联合实体和关系抽取模型[J]. 计算机工程与应用, 2023, 59(13): 164-170.

References

[1] VERGA P，STRUBELL E，MCCALLUM A.Simultaneously self-attending to all mentions for full-abstract biological relation extraction[J].arXiv：1802.10569，2018.
[2] XU K，REDDY S，FENG Y，et al.Question answering on freebase via relation extraction and textual evidence[J].arXiv.1603.00957，2016.
[3] YU M，YIN W，HASAN K S，et al.Improved neural relation detection for knowledge base question answering[J].arXiv：1704.06194，2017.
[4] HU R，ROHRBACH A，DARRELL T，et al.Language-conditioned graph networks for relational reasoning[C]//Proceedings of 2019 IEEE/CVFInternational Conference on Computer Vision（ICCV），2019.
[5] NADEAU D，SEKINE S.A survey of named entity recognition and classification[J].Lingvisticae Investigationes，2007，30（1）：3-26.
[6] MIWA M，SASAKI Y.Modeling joint entity and relation extraction with table representation[C]//Proceedings of 2014 Conference on Empirical Methods in Natural Language Processing，2014.
[7] LI Q，JI H.Incremental joint extraction of entity mentions and relations[C]//Meeting of the Association for Computational Linguistics，2014.
[8] GUPTA P，SCHüTZE H，ANDRASSY B.Table filling multi-task recurrent neural network for joint entity and relation extraction[C]//Proceedings of the 26th International Conference on Computational Linguistics，2016.
[9] KATIYAR A，CARDIE C.Going out on a LIMB：joint extraction of entity mentions and relations without dependency trees[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2017.
[10] ZHENG S，WANG F，BAO H，et al.Joint extraction of entities and relations based on a novel tagging scheme[J].arXiv：1706.05075，2017.
[11] ZENG X，ZENG D，HE S，et al.Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2018.
[12] FU T J，LI P H，MA W Y.GraphRel：modeling text as relational graphs for joint entity and relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019.
[13] BEKOULIS G，DELEU J，DEMEESTER T，et al.Joint entity recognition and relation extraction as a multi-head selection problem[J].Expert Systems with Applications，2018，114：34-45.
[14] HUANG W，CHENG X，WANG T，et al.BERT-based multi-head selection for joint entity-relation extraction[C]//Proceedings of the CCF International Conference on Natural Language Processing and Chinese Computing，2019：713-723.
[15] DEVLIN J，CHANG M W，LEE K，et al.BERT：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[16] LI X，YIN F，SUN Z，et al.Entity-relation extraction as multi-turn question answering[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：1340-1350.
[17] WANG D，TIWARI P，GARG S，et al.Structural block driven enhanced convolutional neural representation for relation extraction[J].Applied Soft Computing，2020，86：105913.
[18] CHIU J P C，NICHOLS E.Named entity recognition with bidirectional LSTM-CNNs[J].arXiv：1511.08308，2015.
[19] LAMPLE G，BALLESTEROS M，SUBRAMANIAN S，et al.Neural architectures for named entity recognition[J].arXiv：1603.01360，2016.
[20] QIAN Y，SANTUS E，JIN Z，et al.GraphIE：a graph-based framework for information extraction[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies（Volume 1：Long and Short Papers），2019：751-761.
[21] LI X，FENG J，MENG Y，et al.A unified MRC framework for named entity recognition[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：5849-5859.
[22] 赵丹丹，黄德根，孟佳娜.多头注意力与字词融合的中文命名实体识别[J].计算机工程与应用，2022，58（7）：142-149.
ZHAO D D，HUANG D G，MENG J N.Chinese named entity recognition by integrating multi-heads attention mechanism and character and words fusion[J].Computer Engineering and Applications，2022，58（7）：142-149.
[23] XU K，FENG Y，HUANG S，et al.Semantic relation classification via convolutional neural networks with simple negative sampling[J].arXiv：1506.07650，2015.
[24] XU Y，MOU L，LI G，et al.Classifying relations via long short term memory networks along shortest dependency paths[J].arXiv：1508.03720，2015.
[25] PENG N，POON H，QUIRK C，et al.Cross-sentence nary relation extraction with graph LSTMs[J].arXiv：1708.03743，2017.
[26] GUPTA P，RAJARAM S，SCHüTZE H，et al.Neural relation extraction within and across sentence boundaries[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[27] WANG J，LU W.Two are better than one：joint entity and relation extraction with table-sequence encoders[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing（EMNLP），2020：1706-1721.
[28] 黄梅根，刘佳乐，刘川.基于BERT的中文多关系抽取方法研究[J].计算机工程与应用，2021，57（21）：234-240.
HUANG M G，LIU J L，LIU C.Research on improved BERT’s Chinese multi-relation extraction method[J].Computer Engineering and Applications，2021，57（21）：234-240.
[29] REN X，WU Z，HE W，et al.CoType：joint extraction of typed entities and relations with knowledge bases[J].arXiv：1610.08763，2016.
[30] MIWA M，BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2016：1105-1116.
[31] ADEL H，SCHüTZE H.Global normalization of convolutional neural networks for joint entity and relation classification[J].arXiv：1707.07719，2017.
[32] 朱艳辉，李飞，胡骏飞.基于三支决策的两阶段实体关系抽取研究[J].计算机工程与应用，2018，54（9）：145-150.
ZHU Y H，LI F，HU J F.Research on two-stage entity relation extraction based on three?way decisions[J].Computer Engineering and Applications，2018，54（9）：145-150.
[33] WANG S，ZHANG Y，CHE W，et al.Joint extraction of entities and relations based on a novel graph scheme[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence，2018.
[34] WALKER C，STRASSEL S，KAZUAKI MAEDA J M.ACE 2005 multilingual training corpus[Z].2006.
[35] BEKOULIS G，DELEU J，DEMEESTER T，et al.Adversarial training for multi-context joint entity and relation extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：2830-2836.
[36] PASZKE A，GROSS S，MASSA F，et al.PyTorch：an imperative style，high-performance deep learning library[C]//Advances in Neural Information Processing Systems，2019.
[37] LUAN Y，WADDEN D，HE L，et al.A Sanh_hierarchical_2019 using dynamic span graphs[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies（Volume 1：Long and Short Papers），2019：3036-3046
[38] DIXIT K，AL-ONAIZAN Y.Span-level model for relation extraction[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：5308-5314.
[39] WADDEN D，WENNBERG U，LUAN Y，et al.Entity，relation，and event extraction with contextualized span representations[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，2019：5784-5789.
[40] ZHAO T，YAN Z，CAO Y，et al.Asking effective and diverse questions：a machine reading comprehension based framework for joint entity-relation extraction[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence，2020：3948-3954.
[41] FEI H，REN Y，JI D.Boundaries and edges rethinking：an end-to-end neural model for overlapping entity relation extraction[J].Information Processing & Management，2020，57（6）：102311.
[42] ZHAO S，HU M，CAI Z，et al.Dynamic modeling cross-modal interactions in two-phase prediction for entity-relation extraction[J].IEEE Transactions on Neural Networks and Learning Systems，2021，34（3）：1122-1131.