结合路径掩蔽和双解码器的图自编码器框架

doi:10.3778/j.issn.1002-8331.2308-0124

摘要/Abstract

摘要： 图自编码器作为一种自监督学习方法，在图神经网络领域得到广泛应用。然而，最近的研究表明，现有的图自编码器通常重构整个图结构，容易产生过拟合数据。此外，这些方法过度强调邻居信息而忽略了结构信息，导致在节点分类任务中表现不佳。针对以上问题，提出了基于路径掩蔽和双解码器的图自编码器框架，用于图表示学习。通过路径掩蔽方法扰动输入图，避免产生过拟合数据。将图神经网络作为编码器，在剩余的图结构上进行消息传递，提高了对图数据的学习能力。提出双解码器对掩蔽边重构，既包含了邻居信息又捕获了结构信息。模型在5个公开的图数据集上进行了实验，并与当前具有代表性的图表示学习方法进行了对比。实验结果表明，提出的方法在5个数据集上均取得了相似或更好的效果，并且在链接预测和节点分类任务上优于基线方法。

关键词: 图自编码器, 自监督学习, 图神经网络, 链接预测, 节点分类

Abstract: Graph autoencoder, as a self-supervised learning method, has been widely applied in the field of graph neural networks. However, recent studies have shown that existing graph autoencoders often reconstruct the entire graph structure, leading to overfitting issues. Additionally, these methods tend to overemphasize neighbor information while neglecting structural information, resulting in poor performance on node classification tasks. To address these issues, a graph autoencoder framework based on path masking and dual decoders is proposed for graph representation learning. Firstly, the input graph is perturbed using path masking to avoid generating overfitting data. Secondly, a graph neural network is employed as the encoder to perform message passing on the remaining graph structure, enhancing the learning capability for graph data. Finally, dual decoders are introduced to reconstruct the masked edges, capturing both neighbor information and structural information. The model is evaluated on five publicly available graph datasets and compared with state-of-the-art graph representation learning methods. Experimental results demonstrate that the proposed approach achieves similar or better performance on all five datasets and outperforms baseline methods in link prediction and node classification tasks.

Key words: graph autoencoder, self-supervised learning, graph neural network, link prediction, node classification

赵韶辉, 马晓, 王建霞. 结合路径掩蔽和双解码器的图自编码器框架[J]. 计算机工程与应用, 2024, 60(24): 140-148.

ZHAO Shaohui, MA Xiao, WANG Jianxia. Graph Autoencoder Framework Combining Path Masking and Dual Decoder[J]. Computer Engineering and Applications, 2024, 60(24): 140-148.

参考文献

[1] BENGIO Y. Deep learning of representations: looking forward[C]//Proceedings of the International Conference on Statistical Language and Speech Processing, 2013: 1-37.
[2] CHEN T, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[C]//Proceedings of the International Conference on Machine Learning, 2020: 1597-1607.
[3] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[4] CHEN L, LI J, PENG Q, et al. Understanding structural vulnerability in graph convolutional networks[J]. arXiv:2108.
06280, 2021.
[5] VELI?KOVI? P, FEDUS W, HAMILTON W L, et al. Deep graph infomax[J]. arXiv:1809.10341, 2018.
[6] PAN S, HU R, LONG G, et al. Adversarially regularized graph autoencoder for graph embedding[J]. arXiv:1802.04407, 2018.
[7] TAN Q, ZHANG X, LIU N, et al. Bring your own view: graph neural networks for link prediction with personalized subgraph selection[C]//Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 2023: 625-633.
[8] QIU J, CHEN Q, DONG Y, et al. GCC: graph contrastive coding for graph neural network pre-training[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020: 1150-1160.
[9] THAKOOR S, TALLEC C, AZAR M G, et al. Large-scale representation learning on graphs via bootstrapping[J]. arXiv:2102.06514, 2021.
[10] YOU Y, CHEN T, SUI Y, et al. Graph contrastive learning with augmentations[C]//Advances in Neural Information Processing Systems, 2020: 5812-5823.
[11] KIPF T N, WELLING M. Variational graph auto-encoders[J]. arXiv:1611.07308, 2016.
[12] XU D, CHENG W, LUO D, et al. InfoGCL: information-aware graph contrastive learning[C]//Advances in Neural Information Processing Systems, 2021: 30414-30425.
[13] HASSANI K, KHASAHMADI A H. Contrastive multi-view representation learning on graphs[C]//Proceedings of the International Conference on Machine Learning, 2020: 4116-4126.
[14] LEE N, LEE J, PARK C. Augmentation-free self-supervised learning on graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 7372-7380.
[15] YOU Y, CHEN T, SHEN Y, et al. Graph contrastive learning automated[C]//Proceedings of the International Conference on Machine Learning, 2021: 12121-12132.
[16] WU L, LIN H, TAN C, et al. Self-supervised learning on graphs: contrastive, generative, or predictive[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 35(4): 4216-4235.
[17] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv:1609.02907, 2016.
[18] TAN Q, LIU N, HUANG X, et al. MGAE: masked autoencoders for self-supervised learning on graphs[J]. arXiv:2201.
02534, 2022.
[19] MENG Z, LIANG S, BAO H, et al. Co-embedding attributed networks[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, 2019: 393-401.
[20] HE K, CHEN X, XIE S, et al. Masked autoencoders are scalable vision learners[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 16000-16009.
[21] CHEN T, SUI Y, CHEN X, et al. A unified lottery ticket hypothesis for graph neural networks[C]//Proceedings of the International Conference on Machine Learning, 2021: 1695-1706.
[22] PEROZZI B, AL-RFOU R, SKIENA S. Deepwalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014: 701-710.
[23] GROVER A, LESKOVEC J. node2vec: scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016: 855-864.
[24] HAMILTON W, YING Z, LESKOVEC J. Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems, 2017.
[25] VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]//Proceedings of the International Conference on Learning Representations, 2017.
[26] TAN Q, LIU N, HUANG X, et al. S2GAE: self-supervised graph autoencoders are generalizable learners with graph masking[C]//Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining, 2023: 787-795.
[27] YING R, HE R, CHEN K, et al. Graph convolutional neural networks for web-scale recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018: 974-983.
[28] HOU Z, LIU X, CEN Y, et al. Graphmae: self-supervised masked graph autoencoders[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022: 594-604.
[29] XU K, HU W, LESKOVEC J, et al. How powerful are graph neural networks?[J]. arXiv:1810.00826, 2018.
[30] LI J, WU R, SUN W, et al. MaskGAE: masked graph modeling meets graph autoencoders[J]. arXiv:2205.10053, 2022.
[31] MAVROMATIS C, KARYPIS G. Graph infoclust: maximizing coarse-grain mutual information in graphs[C]//Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2021: 541-553.
[32] HU Z, DONG Y, WANG K, et al. GPT-GNN: generative pre-training of graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020: 1857-1867.
[33] ZHU Q, DU B, YAN P. Self-supervised training of graph convolutional networks[J]. arXiv:2006.02380, 2020.