Temporal Aware Approach for Dynamic Knowledge Graph Completion

doi:10.3778/j.issn.1002-8331.2112-0057

Abstract

Abstract: Most current approaches for dynamic knowledge graph completion embed temporal dimension into entities or relations. In other words, quaternions are reduced into triples to be completed in the theory of static knowledge graph completion. However, static knowledge graph completion approaches usually focus on entities and relations, which ignore the role of temporal information in the quaternions. Meanwhile, there are sparsity and irregularity in the temporal representation within the knowledge base. To address the above problems, a temporal aware encoder and a temporal convolutional decoder are proposed in this paper. The temporal aware encoder embeds temporal entities, relations and times as same scale vectors and extracts features of quaternions by an improved graph convolutional networks. The temporal convolutional decoder is improved by convolutional neural networks. And it evaluates featrures of the quaternions extracted by the encoder for link prediction. Proposed approach provides more accurate features of time dimension and improves the performance of the completion in temporal knowledge graph. Experiments verify the effectiveness of the proposed work on ICEWS14, ICEWS05-15, Wikidata12k and YAGO11k datasets. Even more, the proposed approach makes a significant improvement in link prediction.

Key words: temporal knowledge graph completion, link prediction, graph convolutional networks, attentional mechanism

摘要： 针对动态知识图谱的补全方法大多将时间维度内嵌于实体或关系中，将四元组降维成三元组后以静态知识图谱补全理论进行补全。静态补全方法通常只对实体关系建模，忽略了时间信息在四元组中的重要作用。同时知识库内时间表述存在稀疏性和不规则性。针对以上问题，提出了时序感知编码器和时序卷积解码器。时序感知编码器将时间维度同实体和关系嵌入为同规模向量，通过改进的图卷积神经网络实现四元组的特征提取。针对时序编码器特征提取后的四元组向量，时序卷积解码器利用卷积神经网络评估全局关系以进行链接预测。所提出的方法可以提供更精确的时间维度特征，提升补全时序图谱的性能。在ICEWS14、ICEWS05-15、Wikidata12k和YAGO11k数据集上的实验验证了提出方法的有效性，同时链接预测效果较优。

关键词: 动态知识图谱补全, 链接预测, 图卷积神经网络, 注意力机制

LI Fengying, FAN Weihao. Temporal Aware Approach for Dynamic Knowledge Graph Completion[J]. Computer Engineering and Applications, 2022, 58(15): 202-209.

李凤英, 范伟豪. 基于时序感知的动态知识图谱补全方法[J]. 计算机工程与应用, 2022, 58(15): 202-209.

References

[1] XIONG C，POWER R，CALLAN J.Explicit semantic ranking for academic search via knowledge graph embedding[C]//Proceedings of the 26th International Conference on World Wide Web，Perth，April 3-7，2017：1271-1279.
[2] ZHANG F，YUAN N J，LIAN D，et al.Collaborative knowledge base embedding for recommender systems[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，San Francisco，August 13-17，2016：353-362.
[3] HAO Y，ZHANG Y，LIU K，et al.An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics，Vancouver，July 30-August 4，2017：221-231.
[4] BOLLACKER K，EVANS C，PARITOSH P，et al.Freebase：a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data，Vancouver，June 10-12，2008：1247-1250.
[5] SUCHANEK F M，KASNECI G，WEIKUM G.Yago：a core of semantic knowledge[C]//Proceedings of the 16th International Conference on World Wide Web，Banff，May 8-12，2007：697-706.
[6] AUER S，BIZER C，KOBILAROV G，et al.Dbpedia：a nucleus for a web of open data[M]//The semantic Web.Berlin，Heidelberg：Springer，2007：722-735.
[7] NAKASHOLE N，WEIKUM G，SUCHANEK F.PATTY：a taxonomy of relational patterns with semantic types[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning，Jeju Island，July 12-14，2012：1135-1145.
[8] NIU F，ZHANG C，Ré C，et al.Elementary：large-scale knowledge-base construction via machine learning and statistical inference[J].International Journal on Semantic Web and Information Systems，2012，8（3）：42-73.
[9] BORDES A，USUNIER N，GARCIA-DURAN A，et al.Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th Advances in Neural Information Processing Systems，Lake Tahoe，December 5-8，2013：2787-2795.
[10] SUN Z，DENG Z H，NIE J Y，et al.Rotate：knowledge graph embedding by relational rotation in complex space[C]//Proceedings of the 7th International Conference on Learning Representations，New Orleans，May 6-9，2019：1-18
[11] LIN Y，LIU Z，LUAN H，et al.Modeling relation paths for representation learning of knowledge bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing，Lisbon，September 17-21，2015：705-714.
[12] NATHANI D，CHAUHAN J，SHARMA C，et al.Learning attention-based embeddings for relation prediction in knowledge graphs[C]//Proceedings of the 57th Association for Computational Linguistics，Florence，July 28- August 2，2019：4710-4723.
[13] YANG B，YIH W，HE X，et al.Embedding entities and relations for learning and inference in knowledge bases[C]//Proceeding of the 3rd International Conference on Learning Representations，San Diego，May 7-9，2015：1-12.
[14] TROUILLON T，WELBL J，RIEDEL S，et al.Complex embeddings for simple link prediction[C]//Proceedings of the 33nd International Conference on Machine Learning，New York，June 19-24，2016：2071-2080.
[15] NGUYEN D Q，NGUYEN T D，NGUYEN D Q，et al.A novel embedding model for knowledge base completion based on convolutional neural network[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，New Orleans，June 1-6，2018：327-333.
[16] DASGUPTA S S，RAY S N，TALUKDAR P.Hyte：Hyperplane-based temporally aware knowledge graph embedding[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，Brussels，October 31-November 4，2018：2001-2011.
[17] GARCíA-DURáN A，DUMAN?I? S，NIEPERT M.Learning sequence encoders for temporal knowledge graph completion[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，Brussels，October 31-November 4，2018：4816-4821.
[18] ZHANG J，LIANG S，DENG Z，et al.Spatial-temporal attention network for temporal knowledge graph completion[C]//Proceedings of 26th Conference on Database Systems for Advanced Applications，Taipei，China，April 11-14，2021：207-223.
[19] LEBLAY J，CHEKOL M W.Deriving validity time in knowledge graph[C]//Companion Proceedings of the Web Conference 2018，Lyon，April 23-27，2018：1771-1776.
[20] LACROIX T，OBOZINSKI G，USUNIER N.Tensor decompositions for temporal knowledge base completion[C]//Proceeding of the 8th International Conference on Learning Representations，Addis Ababa，April 26-30，2020：1-12.
[21] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations，Toulon，April 24-26，2017：1-14.
[22] SCHLICHTKRULL M，KIPF T N，BLOEM P，et al.Modeling relational data with graph convolutional networks[C]//Proceedings of the 15th European Semantic Web Conference，Heraklion，June 3-7，2018：593-607.
[23] 申宇铭，杜剑峰.时态知识图谱补全的方法及其进展[J].大数据，2021，7（3）：30-41.
SHEN Y M，DU J F.Temoral knowledge graph completion：methods and progress[J].Big Data Research，2021，7（3）：30-41.
[24] TAY Y，LUU A T，HUI S C.Non-parametric estimation of multiple embeddings for link prediction on dynamic knowledge graphs[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence，San Francisco，February 4-9，2017：1243-1249.
[25] SANKAR A，WU Y，GOU L，et al.Dysat：deep neural representation learning on dynamic graphs via self-attention networks[C]//Proceedings of the 13th International Conference on Web Search and Data Mining，Houston，February 3-7，2020：519-527.
[26] XU C，CHEN Y Y，NAYYERI M，et al.Temporal knowledge graph completion using a linear temporal regularizer and multivector embeddings[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2021：2569-2578.
[27] VELI?KOVI? P，CUCURULL G，CASANOVA A，et al.Graph attention networks[C]//Proceedings of the 2018 Conference of International Conference on Learning Representations，Vancouver，April 30-May 3，2018：1-12.