Incorporating Relational Awareness and Temporal Attention for Temporal Knowledge Graph Completion

doi:10.3778/j.issn.1002-8331.2210-0266

Abstract

Abstract: To address the problem that most of the existing temporal knowledge graph completion methods embed time information into triples and rely on static knowledge graph completion means to learn entity features, which cannot holistically consider structural information and temporal information in the graph. This paper proposes a temporal knowledge graph completion model incorporating relational awareness and temporal attention（RATA）. On the one hand, by introducing graph convolutional network with relation-aware aggregation mechanism to integrate entity and relation features, relation-specific parameters can enhance the expression ability of the message function, and encapsulate richer neighborhood context information. On the other hand, it employs the long short-term memory network incorporating self-attention mechanism to learn global and local features in the sequence. The experimental results on ICEWS18, ICEWS14, YAGO and WIKI show RATA generally outperforms the baseline model on MRR, Hits@1, Hits@3, and Hits@10, and has better advantages on large-scale temporal datasets.

Key words: temporal knowledge graph, graph convolutional network, long short-term memory, attention mechanism

摘要： 针对现有时序知识图谱补全方法大多将时间信息内嵌于三元组中，依赖静态知识图谱补全方法学习实体特征，无法全面考虑图谱结构信息和时序信息的问题，提出一种融合关系感知与时间注意的时序知识图谱补全方法（incorporating relational awareness and temporal attention for temporal knowledge graph completion，RATA）。一方面，通过引入关系感知聚合机制的图卷积神经网络集成实体和关系特征，特定于关系的参数可以增强消息函数的表达能力，封装更加丰富的邻域上下文信息；另一方面，使用融合自注意力机制的长短期记忆网络学习蕴含在时序数据中的全局特征和局部特征。在ICEWS18、ICEWS14、YAGO和WIKI数据集上的实验结果表明，RATA模型在MRR、Hits@1、Hits@3和Hits@10上普遍优于基线模型，在大规模时序数据集上具有优势。

关键词: 时序知识图谱, 图卷积神经网络, 长短期记忆网络, 注意力机制

XU Zhihong, MAO Chen, WANG Liqin, DONG Yongfeng. Incorporating Relational Awareness and Temporal Attention for Temporal Knowledge Graph Completion[J]. Computer Engineering and Applications, 2023, 59(17): 266-274.

许智宏, 毛琛, 王利琴, 董永峰. 融合关系感知与时间注意的时序知识图谱补全[J]. 计算机工程与应用, 2023, 59(17): 266-274.

References

[1] 李凤英，范伟豪.基于时序感知的动态知识图谱补全方法[J].计算机工程与应用，2022，58（15）：202-209.
LI F Y，FAN W H.Temporal aware approach for dynamic knowledge graph completion[J].Computer Engineering and Applications，2022，58（15）：202-209.
[2] 申宇铭，杜剑峰.时态知识图谱补全的方法及其进展[J].大数据，2021，7（3）：30-41.
SHEN Y M，DU J F.Temporal knowledge graph completion：methods and progress[J].Big Data Research，2021，7（3）：30-41.
[3] DASGUPTA S S，RAY S N，TALUKDAR P.Hyte：hyperplane-based temporally aware knowledge graph embedding[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：2001-2011.
[4] XU C，NAYYERI M，ALKHOURY F，et al.TeRo：a time-aware knowledge graph embedding via temporal rotation[C]//Proceedings of the 28th International Conference on Computational Linguistics，2020：1583-1593.
[5] SANKAR A，WU Y，GOU L，et al.Dysat：deep neural representation learning on dynamic graphs via self-attention networks[C]//Proceedings of the 13th International Conference on Web Search and Data Mining，2020：519-527.
[6] JIN W，QU M，JIN X，et al.Recurrent event network：autoregressive structure inference over temporal knowledge graphs[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing，2020：6669-6683.
[7] SCHLICHTKRULL M，KIPF T N，BLOEM P，et al.Modeling relational data with graph convolutional networks[C]//European Semantic Web Conference.Cham：Springer，2018：593-607.
[8] BORDES A，USUNIER N，GARCIA-DURAN A，et al.Translating embeddings for modeling multi-relational data[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems，San Diego，2013：2787-2795.
[9] TROUILLON T，DANCE C R，GAUSSIER é，et al.Knowledge graph completion via complex tensor factorization[J].The Journal of Machine Learning Research，2017，18（1）：4735-4772.
[10] SUN Z，DENG Z H，NIE J Y，et al.RotatE：knowledge graph embedding by relational rotation in complex space[C]//Proceedings of the 6th International Conference on Learning Representations，Vancouver，April 30-May 3，2018.
[11] LACROIX T，OBOZINSKI G，USUNIER N.Tensor decompositions for temporal knowledge base completion[C]//Proceedings of the 8th International Conference on Learning Representations，Addis Ababa，April 26-30，2020.
[12] SADEGHIAN A，ARMANDPOUR M，COLAS A，et al.ChronoR：rotation based temporal knowledge graph embedding[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2021：6471-6479.
[13] PAREJA A，DOMENICONI G，CHEN J，et al.EvolveGCN：evolving graph convolutional networks for dynamic graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：5363-5370.
[14] WU J，CAO M，CHEUNG J，et al.TeMP：temporal message passing for temporal knowledge graph completion[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing，2020：5730-5746.
[15] BAI L，MA X，ZHANG M，et al.Tpmod：a tendency-guided prediction model for temporal knowledge graph completion[J].ACM Transactions on Knowledge Discovery from Data，2021，15（3）：1-17.
[16] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//31st Annual Conference on Neural Information Processing Systems，Long Beach，2017：5998-6008.
[17] ZHANG W，PAUDEL B，ZHANG W，et al.Interaction embeddings for prediction and explanation in knowledge graphs[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining，2019：96-104.
[18] YANG J，ZHOU W，WEI L，et al.RE-GCN：relation enhanced graph convolutional network for entity alignment in heterogeneous knowledge graphs[C]//International Conference on Database Systems for Advanced Applications.Cham：Springer，2020：432-447.
[19] HAN Z，CHEN P，MA Y，et al.Explainable subgraph reasoning for forecasting on temporal knowledge graphs[C]//Proceedings of the 9th International Conference on Learning Representations，May 3-7，2021.
[20] NICKEL M，TRESP V，KRIEGEL H P.A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on International Conference on Machine Learning，2011：809-816.
[21] YANG B，YIH S W，HE X，et al.Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of the International Conference on Learning Representations，San Diego，May 7-9，2015.
[22] VASHISHTH S，SANYAL S，NITIN V，et al.Composition-based multi-relational graph convolutional networks[C]//Proceedings of the 8th International Conference on Learning Representations，Addis Ababa，April 26-30，2020：1-15.
[23] LEBLAY J，CHEKOL M W.Deriving validity time in knowledge graph[C]//Proceedings of the Web Conference 2018，2018：1771-1776.
[24] GOEL R，KAZEMI S M，BRUBAKER M，et al.Diachronic embedding for temporal knowledge graph completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：3988-3995.
[25] XU C，CHEN Y Y，NAYYERI M，et al.Temporal knowledge graph completion using a linear temporal regularizer and multivector embeddings[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2021：2569-2578.