Advancements in Embedded Static Knowledge Graph Completion Research

doi:10.3778/j.issn.1002-8331.2310-0221

Abstract

Abstract: Knowledge graphs are widely used and semantically rich data representations, which is increasingly becoming a crucial technology in the field of knowledge engineering. However, real-world knowledge graphs often suffer from incompleteness and ambiguity, hindering their application performance. Knowledge graph completion techniques aim to enrich the content of knowledge graphs by predicting missing entities or relations, has been a hot research topic in recent years. In particular, embedding-based approaches have made remarkable progress in knowledge graph completion tasks. It reviews recent embedding-based static knowledge graph completion methods, categorizing them based on approaches such as translation-based models, tensor factorization, neural network models, and pre-trained language models. These methods achieve an improved semantic representation and inferential capabilities by embedding entity relations into continuous vector spaces. At the same time, it has potential advantages in capturing complex relationships between entities and utilizing graph structural information.

Key words: knowledge graph embedding, knowledge graph completion, pre-trained language models

摘要： 知识图谱是一种应用广泛且语义丰富的数据表示形式，日益成为知识工程领域的重要技术。但是由于现实世界中的知识图谱往往存在不完整和含糊的信息，阻碍了知识图谱应用性能。知识图谱补全技术旨在通过预测缺失的实体或关系来丰富知识图谱的内容，是近年来研究的热点，特别是在知识图谱补全任务中采用嵌入式方法取得了显著进展。回顾近年来嵌入式静态知识图谱补全方法，从空间平移、张量分解、神经网络模型、预训练语言模型等角度开展分类探讨。这些方法通过将实体关系嵌入到连续向量空间中，实现了更好的语义表示和推理能力；同时，在捕捉实体间复杂关系、利用图结构信息等方面具有潜在优势。

关键词: 知识图谱嵌入, 知识图谱补全, 预训练语言模型

WU Yujie, XI Xuefeng, CUI Zhiming. Advancements in Embedded Static Knowledge Graph Completion Research[J]. Computer Engineering and Applications, 2024, 60(12): 34-47.

吴玉洁, 奚雪峰, 崔志明. 嵌入式静态知识图谱补全研究进展[J]. 计算机工程与应用, 2024, 60(12): 34-47.

References

[1] WOODS W A. What’s in a link: foundations for semantic networks[M]//Representation and understanding. San Francisco: Morgan Kaufmann, 1975: 35-82.
[2] BERNERS-LEE T J. Information management: a proposal:CERN-DD-89-001-OC[EB/OL]. (1989)[2023-10-16]. https://www.w3.org/History/1989/proposal.html.
[3] SINGHAL A. Introducing the knowledge graph: things, not strings[J]. Official Google Blog, 2012(5): 1-8.
[4] DONG X, GABRILOVICH E, HEITZ G, et al. Knowledge vault: a Web?scale approach to probabilistic knowledge fusion[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014: 601-610.
[5] MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009: 1003-1011.
[6] HUANG X, ZHANG J, LI D, et al. Knowledge graph embedding based question answering[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, 2019: 105-113.
[7] LIN B Y, CHEN X, CHEN J, et al. KagNet: knowledge-aware graph networks for commonsense reasoning[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019: 2829-2839.
[8] WANG H, ZHANG F, XIE X, et al. DKN: deep knowledge-aware network for news recommendation[C]//Proceedings of the 2018 World Wide Web Conference, 2018: 1835-1844.
[9] 官赛萍, 靳小龙, 贾岩涛, 等. 面向知识图谱的知识推理研究进展[J]. 软件学报, 2018, 29(10): 2966-2994.
GUAN S P, JIN X L, JIA Y T, et al. Knowledge reasoning over knowledge graph: a survey[J]. Journal of Software, 2018, 29(10): 2966-2994.
[10] ZHANG W, CHEN J, LI J, et al. Knowledge graph reasoning with logics and embeddings: survey and perspective[J]. arXiv:2202.07412, 2022.
[11] ZEHRA M S, MOHSIN S, WASI S, et al. Financial knowledge graph based financial report query system[J]. IEEE Access, 2021, 9: 69766-69782.
[12] LI L, WANG P, YAN J, et al. Real-world data medical knowledge graph: construction and applications[J]. Artificial Intelligence in Medicine, 2020, 103: 101817.
[13] ABU-SALIH B, AL-TAWIL M, ALJARAH I, et al. Relational learning analysis of social politics using knowledge graph embedding[J]. Data Mining and Knowledge Discovery, 2021, 35(4): 1497-1536.
[14] CHEN J, GENG Y, CHEN Z, et al. Zero-shot and few-shot learning with knowledge graphs: a comprehensive survey[J].Proceedings of the IEEE, 2023, 111(6): 653-685.
[15] 张天成, 田雪, 孙相会, 等.知识图谱嵌入技术研究综述[J].软件学报, 2023, 34(1): 277-311.
ZHANG T C, TIAN X, SUN X H, et al. Overview on knowledge graph embedding technology research[J]. Journal of Software, 2023, 34(1): 277-311.
[16] WANG Q, MAO Z, WANG B, et al. Knowledge graph embedding: a survey of approaches and applications[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(12): 2724-2743.
[17] CHRISTOPHIDES V, EFTHYMIOU V, STEFANIDIS K. Entity resolution in the Web of data[M]. San Rafael: Morgan & Claypool, 2015.
[18] JI S X, PAN S R, CAMBRIA E, et al. A survey on knowledge graphs: representation, acquisition, and applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 33(2): 494-514.
[19] MILLER G A. WordNet: a lexical database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[20] BORDES A, GLOROT X, WESTON J, et al. A semantic matching energy function for learning with multi-relational data: application to word-sense disambiguation[J]. Machine Learning, 2014, 94: 233-259.
[21] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[22] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, 2008: 1247-1250.
[23] ANTOINE B, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems, 2013: 2787-2795.
[24] TOUTANOVA K, CHEN D. Observed versus latent features for knowledge base and text inference[C]//Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality, 2015: 57-66.
[25] SUCHANEK F M, KASNECI G, WEIKUM G. YAGO: a core of semantic knowledge[C]//Proceedings of the 16th International Conference on World Wide Web, 2007: 697-706.
[26] ZHANG Z, CAI J, ZHANG Y, et al. Learning hierarchy-aware knowledge graph embeddings for link prediction[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 3065-3072.
[27] WANG X Z, GAO T Y, ZHU Z C, et al. KEPLER: a unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021, 9:176-194.
[28] WANG Z, ZHANG J, FENG J, et al.Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, 2014: 1112-1119.
[29] LIN Y, LIU Z, SUN M, et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence, 2015: 2181-2187.
[30] JI G, HE S, XU L, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2015: 687-696.
[31] SUN Z, DENG Z H, NIE J Y, et al. RotatE: knowledge graph embedding by relational rotation in complex space[C]//Proceedings of the 7th International Conference on Learning Representations, 2019.
[32] NICKEL M, ROSASCO L, POGGIO T. Holographic embeddings of knowledge graphs[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence, 2016: 1955-1961.
[33] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the International Conference on Machine Learning, New York, NY, USA, June 19-24, 2016: 2071-2080.
[34] XIAO H, HUANG M, ZHU X. From one point to a manifold: knowledge graph embedding for precise link prediction[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2016: 1315-1321.
[35] EBISU T, ICHISE R. TorusE: knowledge graph embedding on a lie group[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 2018.
[36] FAN M, ZHOU Q, CHANG E, et al. Transition-based knowledge graph embedding with relational mapping properties[C]//Proceedings of the 28th Pacific Asia Conference on Language, Information and Computing, 2014: 328-337.
[37] FENG J, HUANG M, WANG M, et al. Knowledge graph embedding by flexible translation[C]//Proceedings of the 15th International Conference on the Principles of Knowledge Representation and Reasoning, 2016: 557-560.
[38] MA S, DING J, JIA W, et al. TransT: type-based multiple embedding representations for knowledge graph completion[C]//Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, Skopje, Macedonia, September 18-22, 2017: 717-733.
[39] MA L, SUN P, LIN Z, et al. Composing knowledge graph embeddings via word embeddings[J]. arXiv:1909.03794,2019.
[40] GE X, WANG Y C, WANG B, et al. Compounding geometric operations for knowledge graph completion[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023: 6947-6965.
[41] NICKEL M, TRESP V, KRIEGEL, et al. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning, Bellevue, WA, USA, June 28-July 2, 2011.
[42] YANG B, YIH W T, HE X, et al. Embedding entities and relations for learning and inference in knowledge bases[J].arXiv:1412.6575, 2014.
[43] ZHANG S, TAY Y, YAO L, et al. Quaternion knowledge graph embeddings[C]//Advances in Neural Information Processing Systems, 2019.
[44] CAO Z, XU Q, YANG Z, et al. Dual quaternion knowledge graph embeddings[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, 2021: 6894-6902.
[45] LIU H, WU Y, YANG Y. Analogical inference for multi-relational embeddings[C]//Proceedings of the 34th International Conference on Machine Learning, 2017: 2168-2178.
[46] BALA?EVI? I, ALLEN C, HOSPEDALES T M. TuckER: tensor factorization for knowledge graph completion[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019: 5185-5194.
[47] TUCKER L R. Some mathematical notes on three-mode factor analysis[J]. Psychometrika, 1966, 31(3): 279-311.
[48] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of the 15th International Semantic Web Conference, Heraklion, Crete, Greece, June 3-7, 2018: 593-607.
[49] LECUN Y, BOTTOU L, BENGIO Y, et al.Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[50] DAI Q N, TU D N, NGUYEN D Q, et al.A novel embedding model for knowledge base completion based on convolutional neural network[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), 2018: 327-333.
[51] JIANG X, WANG Q, WANG B. Adaptive convolution for multi-relational learning[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long and Short Papers), 2019: 978-987.
[52] VASHISHTH S, SANYAL S, NITIN V, et al. InteractE: Improving convolution-based knowledge graph embeddings by increasing feature interactions[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020: 3009-3016.
[53] DEMIR C, NGOMO A C N. Convolutional complex knowledge graph embeddings[C]//Proceedings of the 18th International Semantic Web Conference, June 6-10, 2021: 409-424.
[54] KIPF T, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations, 2017.
[55] TIAN A, ZHANG C, RANG M, et al. RA-GCN: relational aggregation graph convolutional network for knowledge graph completion[C]//Proceedings of the 12th International Conference on Machine Learning and Computing, 2020: 580-586.
[56] SHANG C, TANG Y, HUANG J, et al. End-to-end structure-aware convolutional networks for knowledge base completion[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, 2019: 3060-3067.
[57] CAI L, et al. TransGCN: coupling transformation assumptions with graph convolutional networks for link prediction[C]//Proceedings of the 10th International Conference on Knowledge Capture, 2019: 131-138.
[58] YU D, YANG Y, ZHANG R, et al. Knowledge embedding based graph convolutional network[C]//Proceedings of the Web Conference, 2021: 1619-1628.
[59] VASHISHTH S, SANYAL S, NITIN V, et al. Composition-based multi?relational graph convolutional networks[C]//Proceedings of the 8th International Conference on Learning Representations, 2020.
[60] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]//Proceedings of the International Conference on Learning Representations, 2017.
[61] LIU X, TAN H, CHEN Q, et al. RAGAT: relation aware graph attention network for knowledge graph completion[J]. IEEE Access, 2021, 9: 20840-20849.
[62] CHEN M, ZHANG Y, KOU X, et al. R-GAT: relational graph attention network for multi-relational graphs[J]. arXiv:2109.05922, 2021.
[63] NATHANI D, CHAUHAN J, SHARMA C, et al. Learning attention-based embeddings for relation prediction in knowledge graphs[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019: 4710-4723.
[64] JI K, HUI B, LUO G. Graph attention networks with local structure awareness for knowledge graph completion[J]. IEEE Access, 2020, 8: 224860-224870.
[65] XU H, BAO J, LIU W. Double-branch multi-attention based graph neural network for knowledge graph completion[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023: 15257-15271.
[66] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long and Short Papers), 2019: 4171-4186.
[67] YAO L, MAO C, LUO Y. KG-BERT: BERT for knowledge graph completion[J]. arXiv:1909.03193, 2019.
[68] KIM B, HONG T, KO Y, et al. Multi-task learning for knowledge graph completion with pre?trained language models[C]//Proceedings of the 28th International Conference on Computational Linguistics, 2020: 1737-1743.
[69] WANG B, SHEN T, LONG G, et al. Structure-augmented text representation learning for efficient knowledge graph completion[C]//Proceedings of the Web Conference, 2021: 1737-1748.
[70] WANG L, ZHAO W, WEI Z, et al. SimKGC: simple contrastive knowledge graph completion with pre-trained language models[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022: 4281-4294.
[71] LI D, YANG S, XU K, et al. Multi-task pre-training language model for semantic network completion[J]. ACM Transactions on Asian and Low-Resource Language Information Processing, 2023, 22(11): 1-20.
[72] CHEN C, WANG Y, SUN A, et al. Dipping PLMs sauce: bridging structure and text for effective knowledge graph completion via conditional soft prompting[C]//Findings of the Association for Computational Linguistics, Toronto, Canada, 2023: 11489-11503
[73] CARROLL J D, CHANG J J. Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition[J]. Psychometrika, 1970, 35(3): 283-319.
[74] CAI L, WANG W Y. KBGAN: adversarial learning for knowledge graph embeddings[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2018: 1470-1480.