Survey on Few-Shot Relation Extraction

doi:10.3778/j.issn.1002-8331.2301-0105

Abstract

Abstract: Relation extraction is a vital subtask of information extraction, and it is also an important component in the construction of the knowledge graph. The goal of relation extraction is to extract semantic relationships between entities from the natural language text, so as to discover the connections between the data. The process of relation extraction depends on a large number of labeled training data, though there are often being existed some cold start problems in practical applications. How to learn to extract relation through a small number of samples has become one of the hot topics in this field. Based on the investigation of a large number of literatures, the recent research achievements of the problems on few-shot relation extraction are summarized. Firstly, starting from the definition of few-shot relation extraction tasks, the mechanism of training and the methods of classification are introduced. Secondly, the paper introduces the siamese network, graph neural network and prototypical network from perspective of metric learning. And then the initial network parameters and pre-training network parameters are introduced from the perspective of parameter optimization learning too. Next, the paper introduces the common data sets, evaluation indexes and experimental results of representative methods. Finally, the possible development trends of the future research on few-shot relation extraction are prospected.

Key words: few-shot learning, relation extraction, meta-learning, metric learning, prototypical networks

摘要： 关系抽取是信息抽取的一项重要子任务，也是构建知识图谱的重要环节，其目标是从自然语言文本中抽取出实体之间的语义关系，从而更好地挖掘数据之间的联系。关系抽取的过程需要依赖大量标注的训练样本，而实际应用中却经常存在冷启动问题，如何通过少量样本进行关系抽取已成为该领域关注的热点之一。在调研大量文献的基础上对少样本关系抽取的近期研究现状进行总结，先从少样本关系抽取任务的定义出发，介绍了少样本关系抽取任务的训练机制与分类情况；从度量学习和参数优化学习两个角度分别介绍了基于孪生网络、图神经网络和原型网络，以及基于初始化网络参数和预训练网络参数在少样本关系抽取问题上的研究成果；介绍了少样本关系抽取的常用数据集、评价指标及代表性方法的实验结果；总结了现有研究存在的问题，并展望了少样本关系抽取未来可能的发展趋势。

关键词: 少样本学习, 关系抽取, 元学习, 度量学习, 原型网络

LIU Bei, XU Zhuoming, TAO Wan, LIU Sanmin. Survey on Few-Shot Relation Extraction[J]. Computer Engineering and Applications, 2023, 59(15): 27-37.

刘蓓, 许卓明, 陶皖, 刘三民. 少样本关系抽取研究综述[J]. 计算机工程与应用, 2023, 59(15): 27-37.

References

[1] NIKLAUS C，CETTO M，FREITAS A，et al.A survey on open information extraction[C]//The 28th International Conference on Computational Linguistics，2018：3866-3878.
[2] PAWAR S，PALSHIKAR G K，BHATTACHARYYA P.Relation extraction：a survey[J].arXiv：1712.05191，2017.
[3] CALIFF M E，MOONEY R J.Relational learning of pattern-match rules for information extraction[C]//Proceedings of the 1997 Conference on Computational Natural Language Learning，1997：9-15.
[4] ZHAO S B，GRISHMAN R.Extracting relations with integrated information using kernel methods[C]//The Association for Computational Linguistics，2005：419-426.
[5] GORMLEY M R，YU M，DREDZE M.Improved relation extraction with feature-rich compositional embedding models[C]//Conference on Empirical Methods in Natural Language Processing，2015：1774-1784.
[6] KUMAR S.A survey of deep learning methods for relation extraction[J].arXiv：1705.03645，2017.
[7] 鄂海红，张文静，肖思琪，等.深度学习实体关系抽取研究综述[J].软件学报，2019，30（6）：1793-1818.
YUE H H，ZHANG W J，XIAO S Q，et al.Survey of entity relationship extraction based on deep learning[J] Journal of Software，2019，30（6）：1793-1818.
[8] JI S X，PAN S R，CAMBRIA E，et al.YU：a survey on knowledge graphs：representation，acquisition，and applications[J].IEEE Transactions on Neural Networks and Learning Systems，2022，33（2）：494-514.
[9] ZENG D J，LIU K，LAI S W，et al.Relation classification via convolutional deep neural network[C]//Conference on Computational Linguistics，2014：2335-2344.
[10] NGUYEN T H，GRISHMAN R.Relation extraction：perspective from convolutional neural networks[C]//Proceedings of the 1st Workshop on Vector Space Modeling for Natural Language Processing，2015：39-48.
[11] LIN C，MILLER T A，DLIGACH D，et al.Self-training improves recurrent neural networks performance for temporal relation extraction[C]//Conference on Empirical Methods in Natural Language Processing，2018：165-176.
[12] CAI R，ZHANG X D，WANG H F.Bidirectional recurrent convolutional neural network for relation classification[C]//The Association for Computational Linguistics，2016.
[13] XU Y，MOU L L，LI G，et al.Classifying relations via long short term memory networks along shortest dependency paths[C]//Conference on Empirical Methods in Natural Language Processing，2015：1785-1794.
[14] MIWA M，BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]// The Association for Computational Linguistics，2016.
[15] ZHANG Y H，QI P，MANNING C D.Graph convolution over pruned dependency trees improves relation extraction[C]//Conference on Empirical Methods in Natural Language Processing，2018：2205-2215.
[16] ZHU H，LIN Y K，LIU Z Y，et al.Graph neural networks with generated parameters for relation extraction[C]//The Association for Computational Linguistics，2019：1331-1339.
[17] MINTZ M，BILLS S，SNOW R，et al.Distant supervision for relation extraction without labeled data[C]//The Association for Computational Linguistics/International Joint Conference on Natural Language Processing，2009：1003-1011.
[18] ZENG D J，LIU K，CHEN Y B，et al.Distant supervision for relation extraction via piecewise convolutional neural networks[C]//Conference on Empirical Methods in Natural Language Processing，2015：1753-1762.
[19] QU J F，OUYANG D T，HUA W，et al.Distant supervision for neural relation extraction integrated with word attention and property features[J].Neural Networks，2018，100：59-69.
[20] QIN P D，XU W R，WANG W Y.DSGAN：generative adversarial training for distant su-pervision relation extraction[C]//The Association for Computational Linguistics，2018：496-505.
[21] SMIRNOVA A，CUDRé-MAUROUX P.Relation extraction using distant supervision：a survey[J].ACM Computing Surveysa，2019，51（5）：1-35.
[22] JIA W，DAI D，XIAO X Y，et al.ARNOR：attention regularization based noise reduction for distant supervision relation classification[C]//The Association for Computational Linguistics，2019：1399-1408.
[23] ZHANG X T，SUNG F，QIANG Y T，et al.HOSPEDALES：Deep comparison：relation columns for few-shot learning[J].arXiv：1811.07100，2018.
[24] CHEN W Y，LIU Y C，KIRA Z，et al.A closer look at few-shot classification[C]//International Conference on Learning Representations，2019.
[25] WANG Y Q，YAO Q M，KWOK J T，et al.Generalizing from a few examples：a sur-vey on few-shot learning[J].ACM Computing Surveys，2020，53（3）：1-34.
[26] CHEN Y B，WANG X L，LIU Z，et al.Trevor darrell：a new meta-baseline for few-shot learning[J].arXiv：2003. 04390，2020.
[27] HUISMAN M，VAN RIJN J N，PLAAT A.A survey of deep meta-learning[J].Artificial Intelligence Review，2021，54（6）：4483-4541.
[28] HOSPEDALES T M，ANTONIOU A，MICAELLI P，et al.Meta-learning in neural networks：a survey[J].arXiv：2004.05439，2020.
[29] VINYALS O，BLUNDELL C，LILLICRAP T，et al.Matching networks for one shot learning[C]//Conference and Workshop on Neural Information Processing Systems，2016：3630-3638.
[30] PENG H M.A comprehensive overview and survey of recent advances in meta-learning[J].arXiv：2004.11149，2020.
[31] SABO O，ELAZAR Y，GOLDBERG Y，et al.Revisiting few-shot relation classification：evaluation data and classification schemes[J].Transactions of the Association for Computational Linguistics，2021：9：691-706.
[32] YIN W P.Meta-learning for few-shot natural lan-guage processing：a survey[J].arXiv：2007.09604，2020.
[33] HAN X，ZHU H，YU P F，et al.FewRel：a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation[C]//Conference on Empirical Methods in Natural Language Processing，2018：4803-4809.
[34] HAN X，GAO T Y，LIN Y K，et al.More data，more relations，more context and more openness：a review and outlook for relation extraction[C]//The Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing，2020：745-758.
[35] KOCH G，ZEMEL R，SALAKHUTDINOV R.Siamese neural networks for one-shot image recognition[C]//International Conference on Machine Learning，2015.
[36] WU R D，YAO Y，HAN X，et al.Open relation extraction：relational knowledge transfer from supervised data to unsupervised data[C]//Conference on Empirical Methods in Natural Language Processing/International Joint Conference on Natural Language Processing，2019：219-228.
[37] GAO T Y，HAN X，XIE R B，et al.Neural snowball for few-shot relation learning[C]//Association for the Advancement of Artificial Intelligence，2020：7772-7779.
[38] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[C]//International Conference on Learning Representations，2017.
[39] VELICKOVIC P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[40] GARCIA SATORRAS V，BRUNA ESTRACH J.Few-shot learning with graph neural networks[C]//International Conference on Learning Representations，2018.
[41] LYU C，LIU W J，WANG P.Few-shot text classification with edge-labeling graph neural network-based prototypical network[C]//The Conference on Computational Linguistics，2020：5547-5552.
[42] DING K Z，WANG J L，LI J D，et al.Graph prototypical networks for few-shot learning on attributed networks[C]//Conference on Information and Knowledge Management，2020：295-304.
[43] XIE Y X，XU H，LI J E，et al.Heterogeneous graph neural networks for noisy few-shot relation classification[J].Knowledge-Based Systems，2020，194：105548.
[44] LI W L，QIANT Y.Graph-based model generation for few-shot relation extraction[C]//Conference on Empirical Methods in Natural Language Processing，2022：62-71.
[45] SNELL J，SWERSKY K，ZEMEL R S.Prototypical networks for few-shot learning[C]//Conference and Workshop on Neural Information Processing Systems，2017：4077-4087.
[46] GAO T Y，HAN X，LIU Z Y.Hybrid attention-based prototypical networks for noisy few-shot relation classification[C]//Association for the Advancement of Artificial Intelligence，2019：6407-6414.
[47] FAN M，BAI Y Q，SUN M M，et al.Large margin prototypical network for few-shot relation classification with fine-grained features[C]//Conference on Information and Knowledge Management，2019：2353-2356.
[48] YE Z X，LING Z H.Multi-level matching and aggregation network for few-shot relation classification[C]//The Association for Computational Linguistics，2019：2872-2881.
[49] PANG N，TAN Z，XU H，et al.Boosting knowledge base automatically via few-shot relation classification[J].Frontiers Neurorobotics 14，2020：584192.
[50] REN H P，CAI Y，CHEN X F，et al.A two-phase prototypical network model for incremental few-shot relation classification[C]//The International Conference on Computational Linguistics，2020：1618-1629.
[51] XIAO Y，JIN Y C，HAO K R.Adap-tive prototypical networks with label words and joint representation learning for few-shot relation classification[J].IEEE Transactions on Neural Networks and Learning Systems，2021，34（3）：1406-1417.
[52] YIN G Z，WANG X，ZHANG H L，et al.Cost-effective CNNs-based prototypical networks for few-shot relation classification across domains[J].Knowledge Based System，2022，253：109470.
[53] LI X W，LIU C，YU J，et al.Prototypical attention network for few-shot relation classification with entity-aware embedding module[J].Applied Intelligence，2023，53：10978-10994.
[54] DEVLIN J，CHANG M W，LEE K，et al.BERT：pre-training of deep bidirectional transformers for language understanding[C]//The North American Chapter of the Association for Computational Linguistics，2019：4171-4186.
[55] GAO T Y，HAN X，ZHU H，et al.FewRel 2.0：towards more challenging few-shot relation classification[C]//Conference on Empirical Methods in Natural Language Processing/International Joint Conference on Natural Language Processing，2019：6249-6254.
[56] HUI B，LIU L，CHEN J，et al.Few-shot relation classification by context attention-based prototypical networks with BERT[J].EURASIP Journal on Wireless Communications and Networking，2020（1）：118.
[57] WANG Y Y，BAO J W， LIU G Y，et al.Learning to decouple relations：few-shot relation classification with entity-guided attention and confusion-aware training[C]//The International Conference on Computational Linguistics，2020：5799-5809.
[58] WEN W，LIU Y B，OUYANG C P，et al.Enhanced prototypical network for few-shot relation extraction[J].Information Processing & Management，2021，58（4）：102596.
[59] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Conference and Workshop on Neural Information Processing Systems，2017：5998-6008.
[60] DING N，WANG X B，FU Y，et al.Prototypical representation learning for relation extraction[C]//International Conference on Learning Representations，2021.
[61] CUI L，YANG D Q，YU J X，et al.：Refining sample embeddings with relation prototypes to enhance continual relation extraction[C]//International Joint Conference on Natural Language Processing/International Joint Conference on Natural Language Processing，2021：232-243.
[62] QU M，GAO T Y，XHONNEUX L A C，et al.Few-shot relation extraction via Bayesian meta-learning on relation graphs[C]//International Conference on Machine Learning，2020：7867-7876.
[63] YU H Y，ZHANG N Y，DENG S M，et al.Bridging text and knowledge with multi-prototype embedding for few-shot relational triple extraction[C]//International Conference on Computational Linguistics，2020：6399-6410.
[64] YANG K J，ZHENG N T，DAI X Y，et al.Enhance prototypical network with text descriptions for few-shot relation classification[C]//Conference on Information and Knowledge Management，2020：2273-2276.
[65] ZHANG J W，ZHU J Q，YANG Y，et al.Knowledge-enhanced domain adaptation in few-shot relation classification[C]//ACM Special Interest Group on Knowledge Discovery and Data Mining，2021：2183-2191.
[66] YANG S，ZHANG Y F，NIU G L，et al.Entity concept enhanced few-shot relation extraction[C]//International Joint Conference on Natural Language Processing/International Joint Conference on Natural Language Processing，2021：987-991.
[67] SHALABY W，ZADROZNY W，JIN H X.Beyond word embeddings：learning entity and concept representations from large scale knowledge bases[J].Information Retrieval Journal 2019：22（6）：525-542.
[68] ZHANG P Y，LU W.Better few-shot relation extraction with label prompt dropout[C]//Conference on Empirical Methods in Natural Language Processing，2022：6996-7006.
[69] HE K，HUANG Y C，MAO R，et al.Virtual prompt pre-training for prototype-based few-shot relation extrac-tion[J].Expert Systems with Applications，2023：213（Part）：118927.
[70] SOARES L B，FITZGERALD N，LING J，et al.Matching the blanks：distributional similarity for relation learning[C]//The Association for Computational Linguistics，2019：2895-2905.
[71] PENG H，GAO T Y，HAN X，et al.Learning from context or names? an empirical study on neural relation extraction[C]//Conference on Empirical Methods in Natural Language Processing，2020：3661-3672.
[72] HAN J L，CHENG B，LU W.Exploring task difficulty for few-shot relation extraction[C]//Conference on Empirical Methods in Natural Language Processing，2021：2605-2616.
[73] DONG M Q，PAN C G，LUO Z P.MapRE：an effective semantic mapping approach for low-resource relation extraction[C]//Conference on Empirical Methods in Natural Language Processing，2021：2694-2704.
[74] LIU Y，HU J P，WAN X，et al.Learn from relation information：towards proto-type representation rectification for few-shot relation extraction[C]//NAACL-HLT（Findings），2022：1822-1831.
[75] LIU Y，HU J P，WAN X，et al.A simple yet effective relation information guided approach for few-shot relation extraction[C]//NAACL，2022：757-763.
[76] FINN C，ABBEEL P，LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//International Conference on Machine Learning，2017：1126-1135.
[77] OBAMUYIDE A，VLACHOS A.Model-agnostic meta-learning for relation classification with limited supervision[C]//International Conference on Machine Learning，2019：5873-5879.
[78] DONG B W，YAO Y，XIE R B，et al.Meta-information guided meta-learning for few-shot relation classification[C]//The International Conference on Computational Linguistics，2020：1594-1605.
[79] GENG X Q，CHEN X W，ZHU K Q，et al.MICK：a meta-learning framework for few-shot relation classification with small training data[C]//Conference on Information and Knowledge Management，2020：415-424.
[80] QIAN W，ZHU Y S.Adversarial learning with domain-adaptive pretraining for few-shot relation classification across domains[C]//International Conference on Cloud Computing and Security，2021：134-139.
[81] POPOVIC N，F?RBER M.Few-shot document-level relation extraction[C]//The North American Chapter of the Association for Computational Linguistics，2022：5733-5746.
[82] BIESIALSKA M，BIESIALSKA K，COSTA-JUSSà M R.Continual lifelong learning in natural language processing：a survey[C]//International Conference on Computational Linguistics，2020：6523-6541.
[83] MAZUMDER P，SINGH P，RAI P.Few-shot lifelong learning[C]//Association for the Advancement of Artificial Intelligence，2021：2337-2345.
[84] WANG Y T，LOU R Z，ZHANG K，et al.More：a metric learning based framework for open-domain relation extraction[C]//International Conference on Acoustics，Speech and Signal Processing，2021：7698-7702.