Abstract: In response to the issues in temporal knowledge graph reasoning methods that fail to adequately capture semantic dependencies, temporal evolution information, and lack interpretability, a temporal knowledge graph reasoning method based on hierarchical knowledge embedding and reinforcement learning is proposed, named THKERL. THKERL consists of two key components： the hierarchical knowledge embedding model (HKEM) and the reinforcement learning reasoning model (RLRM). HKEM obtains more accurate knowledge graph feature representations through two levels of knowledge embedding： the subgraph level aims to model the semantic dependencies between concurrent facts in each knowledge graph, while the global graph level is primarily used to capture the dynamic feature information of entities as they evolve over time. On this basis, RLRM employs reinforcement learning, introducing a weighted action scoring mechanism to design the policy network, fully considering the relationship between query questions and reasoning paths for reward shaping, to achieve more reliable knowledge reasoning. To validate the effectiveness of the THKERL method, experiments are conducted on datasets such as ICEWS14, and the experimental results are compared and analyzed with mainstream temporal knowledge graph reasoning methods like TiTer. The experimental results indicate that THKERL achieves an average improvement of over 5.9 percentage points in Hits@[k] and over 6.8 percentage points in MRR in entity prediction tasks.

Key words: temporal knowledge graph, hierarchical knowledge embedding, knowledge reasoning, reinforcement learning, reward shaping

摘要： 针对时序知识图谱推理方法中未能充分捕捉语义依赖、时间演变信息以及缺乏可解释性等问题，提出一种基于分级知识嵌入与强化学习的时序知识图谱推理方法，命名为THKERL。THKERL包含两个关键组件：分级知识嵌入模型（HKEM)以及强化学习推理模型（RLRM)。HKEM通过两个级别知识嵌入以获得更准确的知识图谱特征表示：子图级别旨在建模每个知识图中并发事实之间的语义依赖关系，而全局图级别主要用于捕捉实体随时间演变的动态特征信息。在此基础上，RLRM使用强化学习，引入加权动作评分机制设计策略网络，充分考虑查询问题与推理路径关系进行奖励塑形，以实现更可靠的知识推理。为验证THKERL方法的有效性，在ICEWS14等数据集上进行实验，并将实验结果与TiTer等主流时序知识图谱推理方法进行对比分析。实验结果表明，THKERL在实体预测任务上的Hits@[k]平均提升超过5.9个百分点，MRR平均提升超过6.8个百分点。

关键词: 时序知识图谱, 分级知识嵌入, 知识推理, 强化学习, 奖励塑形