Completion of Temporal Knowledge Graph for Historical Contrastive Learning

doi:10.3778/j.issn.1002-8331.2307-0291

Abstract

Abstract: Aiming at the problem that the existing temporal knowledge graph completion model is highly dependent on events that have occurred in history and the prediction of events that have not occurred in history is inaccurate, a completion of temporal knowledge graph for comparing historical and non-historical information (CHNH) with time series information is proposed. Firstly, the model captures long-term dependencies in the sequence through BiLSTM, ensuring accurate encoding of historical information. Then, the graph convolution operation is performed using RGCN to learn the global graph representation. In the prediction process, different scoring functions are used for separately coded historical and non-historical information to determine the dependence degree of the prediction entity on these two types of information. In this way, the model can more effectively complete entities and relationships, improving the predictive performance of the model. Experimental results on ICEWS18, GDERT and YAGO datasets show that the CHNH model generally outperforms the baseline model in MRR, Hits@1, Hits@3 and Hits@10.

Key words: temporal knowledge graph, historical information, non-historical information, contrastive learning

摘要： 针对现有的时序知识图谱补全模型高度依赖历史上已经发生过的事件，对历史上未发生过的事件预测不够准确的问题，提出了一种加入时序信息的对比历史与非历史信息的时序知识图谱补全模型（completion of temporal knowledge graph for comparing historical and non-historical information，CHNH）。该模型通过BiLSTM捕捉序列中的长期依赖关系，确保准确地编码历史信息。使用RGCN进行图卷积操作，从而学习到全局的图表示。在预测过程中，针对分开编码的历史和非历史信息，采用不同的评分函数来确定预测实体对这两类信息的依赖程度。通过这种方式，模型能够更有效地补全实体和关系，提高模型的预测性能。在ICEWS18、GDELT和YAGO数据集上的实验结果表明，CHNH模型在MRR、Hits@1、Hits@3和Hits@10上普遍优于基线模型。

关键词: 时序知识图谱, 历史信息, 非历史信息, 对比学习

XU Zhihong, QIU Penglin, WANG Liqin, DONG Yongfeng. Completion of Temporal Knowledge Graph for Historical Contrastive Learning[J]. Computer Engineering and Applications, 2024, 60(22): 154-161.

许智宏, 邱鹏林, 王利琴, 董永峰. 基于历史对比学习的时序知识图谱补全[J]. 计算机工程与应用, 2024, 60(22): 154-161.

References

[1] DAI Y, WANG S, XIONG N N, et al. A survey on knowledge graph embedding: approaches, applications and benchmarks[J]. Electronics, 2020, 9(5): 750.
[2] CAI B, XIANG Y, GAO L, et al. Temporal knowledge graph completion: a survey[J]. arXiv:2201.08236, 2022.
[3] 申宇铭, 杜剑峰. 时态知识图谱补全的方法及其进展[J]. 大数据, 2021, 7(3): 30-41.
SHEN Y M, DU J F. Methods and progress of temporal knowledge graph completion[J]. Big Data, 2021, 7(3): 30-41.
[4] WU J. Deep learning for temporal knowledge graph completion[D]. Montréal: McGill University, 2021.
[5] GRAVES A. Long short-term memory[M]//Supervised sequence labelling with recurrent neural networks. Berlin, Heidelberg: Springer, 2012: 37-45.
[6] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of the 15th International Conference on Semantic Web, Heraklion, 2018: 593-607.
[7] BORDES A, USUNIER N, GATCIADURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems 26, 2013.
[8] WANG Z, ZHANG J, FENG J, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, 2014.
[9] FAN M, ZHOU Q, CHANG E, et al. Transition-based knowledge graph embedding with relational mapping properties[C]//Proceedings of the 28th Pacific Asia Conference on Language, Information and Computing, 2014: 328-337.
[10] NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning, 2011: 809-816.
[11] YANG B, YIH W, HE X, et al. Embedding entities and relations for learning and inference in knowledge bases[J]. arXiv:1412.6575, 2014.
[12] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning, 2016: 2071-2080.
[13] SOCHER R, CHEN D, MANNING C D, et al. Reasoning with neural tensor networks for knowledge base completion[C]//Advances in Neural Information Processing Systems 26, 2013.
[14] DASGUPTA S S, RAY S N, TALUKDAR P P. HyTE: hyperplane-based temporally aware knowledge graph embedding[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018: 2001-2011.
[15] HU Z, GUTIERREZ-BASULTO V, XIANG Z, et al. Type-aware embeddings for multi-hop reasoning over knowledge graphs[J]. arXiv:2205.00782, 2022.
[16] SCARSELLI F, GORI M, TSOI A C, et al. The graph neural network model[J]. IEEE Transactions on Neural Networks, 2008, 20(1): 61-80.
[17] ELMAN J L. Finding structure in time[J]. Cognitive Science, 1990, 14(2): 179-211.
[18] XU C, NAYYERI M, ALKHOURY F, et al. TeRo: a time-aware knowledge graph embedding via temporal rotation[J]. arXiv:2010.01029, 2020.
[19] JIN W, QU M, JIN X, et al. Recurrent event network: auto-
regressive structure inference over temporal knowledge graphs[J]. arXiv:1904.05530, 2019.
[20] LEBLAY J, CHEKOL M W. Deriving validity time in knowledge graph[C]//Companion Proceedings of the Web Conference 2018, 2018: 1771-1776.
[21] XU C, NAYYERI M, ALKHOURY F, et al. Temporal knowledge graph embedding model based on additive time series decomposition[J]. arXiv:1911.07893, 2019.
[22] BOSSELUT A, RASHKIN H, SAP M, et al. COMET: commonsense transformers for automatic knowledge graph construction[J]. arXiv:1906.05317, 2019.
[23] BAI L, MA X, ZHANG M, et al. TPmod: a tendency-guided prediction model for temporal knowledge graph completion[J]. ACM Transactions on Knowledge Discovery from Data, 2021, 15(3): 1-17.
[24] CHUNG J, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv:1412.3555, 2014.
[25] ZHU C, CHEN M, FAN C, et al. Learning from history: modeling temporal knowledge graphs with sequential copy-generation networks[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021: 4732-4740.
[26] XU Y, OU J, XU H, et al. Temporal knowledge graph reasoning with historical contrastive learning[C]//Proceedings of the 37th AAAI Conference on Artificial Intelligence, 2023: 4765-4773.
[27] SADEGHIAN A, ARMANDPOUR M, COLAS A, et al. ChronoR: rotation based temporal knowledge graph embedding[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021: 6471-6479.
[28] LEETARU K, SCHRODT P A. GDELT: global data on events, location, and tone, 1979—2012[C]//Annual Meeting of the International Studies Association, 2013.
[29] REBELE T, SUCHANEK F, HOFFART J, et al. YAGO: a multilingual knowledge base from Wikipedia, Wordnet, and Geonames[C]//Proceedings of the 15th International Semantic Web Conference, Kobe, 2016: 177-185.
[30] HAN Z, CHEN P, MA Y, et al. Explainable subgraph reasoning for forecasting on temporal knowledge graphs[C]//Proceedings of the 8th International Conference on Learning Representations, 2020.