利用多头注意力融合数值属性的知识图谱嵌入方法

doi:10.3778/j.issn.1002-8331.2407-0199

摘要/Abstract

摘要： 知识图谱作为结构化的语义知识库通过实体与关系表示现实世界。传统的知识图谱嵌入方法往往忽略了知识图谱中的数值信息，将数值信息通过数值编码的方式纳入知识图谱的链接预测过程，并利用多头注意力机制结合数值信息提升知识图谱链接预测任务的能力。研究通过自监督学习方法对已有的知识图谱数据生成正样本与负样本扩展数据集，提高模型对数据的利用效率。采用对比损失函数与交叉熵损失函数优化模型性能。实验部分选取了数据集Spotify、US-cities与Credit，并在数据集上进行链接预测实验，在MRR指标上相较于现有最佳模型分别提升5.54%、4.03%与1.12%，表明了该研究方法在链接预测任务中的有效性。

关键词: 知识图谱, 链接预测, 多头注意力, 实体嵌入, 自监督学习

Abstract: As a structured semantic knowledge base, the knowledge graph represents the real world through entities and relationships. Traditional knowledge graph embedding methods often ignore the numerical information in the knowledge graph. In this paper, numerical information is incorporated into the link prediction process of knowledge graph through numerical coding, and the multi-head attention mechanism combined with numerical information is used to improve the ability of the link prediction task of knowledge graph. The self-supervised learning method is used to generate positive and negative samples from the existing knowledge graph data to expand the dataset， thereby improving the efficiency of the model’s use of data. The performance of the model is optimized by comparing the loss function and the cross-entropy loss function. In the experimental part, the datasets Spotify, US-cities and Credit are selected, and the link prediction experiments are carried out on the datasets, and the MRR indicators are improved by 5.54%, 4.03% and 1.12% respectively compared with the existing best models, indicating the effectiveness of the research method in the link prediction task.

Key words: knowledge graph, link prediction, multi-head attention, entity embedding, self-supervised learning

冯文龙, 张东, 李冠宇. 利用多头注意力融合数值属性的知识图谱嵌入方法[J]. 计算机工程与应用, 2025, 61(20): 228-237.

FENG Wenlong, ZHANG Dong, LI Guanyu. MHAKGE： Knowledge Graph Embedding Method Using Multi-Head Attention for Fusing Numerical Attributes[J]. Computer Engineering and Applications, 2025, 61(20): 228-237.

参考文献

[1] 刘峤, 李杨, 段宏, 等. 知识图谱构建技术综述[J]. 计算机研究与发展, 2016, 53(3): 582-600.
LIU Q, LI Y, DUAN H, et al. Knowledge graph construction techniques[J]. Journal of Computer Research and Development, 2016, 53(3): 582-600.
[2] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the ACM SIGMOD International Conference on Management of Data. New York: ACM, 2008: 1247-1250.
[3] VRANDE?I? D, KR?TZSCH M. Wikidata[J]. Communic-ations of the ACM, 2014, 57(10): 78-85.
[4] LEHMANN J, ISELE R, JAKOB M, et al. DBpedia a large-scale, multilingual knowledge base extracted from Wikipedia[J]. Semantic Web, 2015, 6(2): 167-195.
[5] SUCHANEK F M, KASNECI G, WEIKUM G. Yago: a core of semantic knowledge[C]//Proceedings of the 16th International Conference on World Wide Web. New York: ACM, 2007: 697-706.
[6] 徐增林, 盛泳潘, 贺丽荣, 等. 知识图谱技术综述[J]. 电子科技大学学报, 2016, 45(4): 589-606.
XU Z L, SHENG Y P, HE L R, et al. Review on knowledge graph techniques[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(4): 589-606.
[7] 张天成, 田雪, 孙相会, 等. 知识图谱嵌入技术研究综述[J]. 软件学报, 2023, 34(1): 277-311.
ZHANG T C, TIAN X, SUN X H, et al. Overview on knowledge graph embedding technology research[J]. Journal of Software, 2023, 34(1): 277-311.
[8] DAI Y F, WANG S P, XIONG N N, et al. A survey on knowledge graph embedding: approaches, applications and benchmarks[J]. Electronics, 2020, 9(5): 750.
[9] WANG M H, QIU L L, WANG X L. A survey on knowledge graph embeddings for link prediction[J]. Symmetry, 2021, 13(3): 485.
[10] TAN Z X, CHEN Z L, FENG S B, et al. KRACL: contrastive learning with graph context modeling for sparse knowledge graph completion[J]. arXiv:2208.07622, 2022.
[11] WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, 2014: 1112-1119.
[12] JI G L, HE S Z, XU L H, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Stroudsburg: ACL, 2015: 687-696.
[13] WANG Q, MAO Z D, WANG B, et al. Knowledge graph embedding: a survey of approaches and applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(12): 2724-2743.
[14] 杜晓娟, 陶以政, 李龚亮. Trans 系列知识表示方法发展研究[J]. 计算机与数字工程, 2023, 51(11): 2515-2517.
DU X J, TAO Y Z, LI G L. Research on development of Trans knowledge representation methods[J]. Computer & Digital Engineering, 2023, 51(11): 2515-2517.
[15] YANG B, YIH W, HE X, et al. Embedding entities and rel-ations for learning and inference in knowledge bases[J]. arXiv:1412.6575, 2014.
[16] TROUILLON T, WELBL J, RIEDEL S, et al. Complex emb-eddings for simple link prediction[C]//Proceedings of the 33rd International Conference on International Conference on Mac-hine Learning, 2016: 2071-2080.
[17] VILNIS L, LI X, MURTY S, et al. Probabilistic embedding of knowledge graphs with box lattice measures[J]. arXiv:1805.06627, 2018.
[18] BALA?EVI? I, ALLEN C, HOSPEDALES T M. TuckER: tensor factorization for knowledge graph completion[J]. arXiv:1901.09590, 2019.
[19] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2018: 1811-1818.
[20] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[J]. arXiv: 1703.06103, 2017.
[21] CAO J X, LIN X X, GUO S, et al. Bipartite graph embedding via mutual information maximization[C]//Proceedings of the 14th ACM International Conference on Web Search and Data Mining. New York: ACM, 2021: 635-643.
[22] LIU C, YU C H, GUI N, et al. SimGCL: graph contrastive learning by finding homophily in heterophily[J]. Knowledge and Information Systems, 2024, 66(3): 2089-2114.
[23] WANG P, AGARWAL K, HAM C, et al. Self-supervised learning of contextual embeddings for link prediction in heterogeneous networks[J]. arXiv:2007.11192, 2020.
[24] KRISTIADI A, KHAN M A, LUKOVNIKOV D, et al. Incorporating literals into knowledge graph embeddings[J]. arXiv:1802.00934, 2018.
[25] GARCíA-DURáN A, NIEPERT M. KBLRN: end-to-end learning of knowledge base representations with latent, relational, and numerical features[J]. arXiv:1709.04676, 2017.
[26] TAY Y, TUAN L A, PHAN M C, et al. Multi-task neural network for non-discrete attribute prediction in knowledge graphs[C]//Proceedings of the ACM on Conference on Information and Knowledge Management. New York: ACM, 2017: 1029-1038.
[27] KIM G, KIM S, KIM K K, et al. Exploiting relation-aware attribute representation learning in knowledge graph embedding for numerical reasoning[C]//Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: ACM, 2023: 1086-1096.
[28] ZHANG Z Q, CAI J Y, ZHANG Y D, et al. Learning hierarchy-aware knowledge graph embeddings for link prediction[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2020: 3065-3072.
[29] ALLEN C, BALAZEVIC I, HOSPEDALES T M. On understanding knowledge graph representation[J]. arXiv:1909.11611, 2019.
[30] BALAZEVIC I, ALLEN C, HOSPEDALES T. Multi-relational poincaré graph embeddings[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems, 2019: 4463-4473.