多级特征增强的图表示学习模型

doi:10.3778/j.issn.1002-8331.2202-0229

摘要/Abstract

摘要： 针对图数据的表示学习在推荐系统、链接预测等图下游任务已展现出重要的研究价值。然而目前主流的方法存在一些缺陷：图卷积网络的固定传播模式限制节点表示的语义表达能力，以及编码器-解码器结构中的正则化重建阻碍学习节点间的差异化特征，这些都可能导致节点表示不能很好适应图下游任务。为此，基于互信息最大化理论提出一种多级特征增强的图表示学习模型，能以无监督的方式生成高质量的节点表示。模型使用提取器保留节点原始属性中的差异化特征，利用注意力聚合器维持编码空间中节点分布的局部相关性和全局差异性，应用深度图信息最大化策略统一全局编码规则。实验结果证明，在几个基准图数据集上该模型在直推式学习和归纳式学习下的编码表现均超过了所有的主流对比基线。

关键词: 图表示学习, 互信息最大化, 无监督学习, 直推式学习, 归纳式学习

Abstract: Representation learning based on graph data has shown significant value for graph downstream tasks such as recommendation system and link prediction. However, current methods have some drawbacks：the fixed propagation of graph neural network limits the semantic expression of node representations, and encoder-decoder architecture with regularized reconstruction is prevented from learning differentiated features between nodes, which may lead to node representations not well suited to some graph downstream tasks. Therefore, a multi-level feature augmented graph representation learning model has been proposed via mutual information maximization, which is capable of learning high quality node representations in an unsupervised manner. This model first uses an extractor to preserve distinguishable features contained in original attributes, which are then fed to an aggregator to maintain the local relevance and global difference of nodes in encoding space. Finally, the strategy of deep graph infomax is applied to unify the global encoding rules. Experimental results demonstrate that the encoding performance of the model completely outperforms all mainstream comparative baselines on several classification benchmark datasets for transductive and inductive learning.

Key words: graph representation learning, mutual information maximization, unsupervised learning, transductive learning, inductive learning

冯耀, 孔兵, 周丽华, 包崇明, 王崇云. 多级特征增强的图表示学习模型[J]. 计算机工程与应用, 2023, 59(11): 131-140.

FENG Yao, KONG Bing, ZHOU Lihua, BAO Chongming, WANG Chongyun. Graph Representation Learning Model for Multi-Level Feature Augmentation[J]. Computer Engineering and Applications, 2023, 59(11): 131-140.

参考文献

[1] REN H，HU W，LESKOVEC J.Query2box：reasoning over knowledge graphs in vector space using box embeddings[J].arXiv：2002.05969，2020.
[2] GAUDELET T，DAY B，JAMASB A R，et al.Utilising graph machine learning within drug discovery and development[J].arXiv：2012.05716，2020.
[3] WANG S，HU L，WANG Y，et al.Graph learning approaches to recommender systems：a review[J].arXiv：2004.11718，2020.
[4] WU S，ZHANG W，SUN F，et al.Graph neural networks in recommender systems：a survey[J].arXiv：2011.02260，2020.
[5] PEROZZI B，AL-RFOU R，SKIENA S.Deepwalk：online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2014：701-710.
[6] GROVER A，LESKOVEC J.node2vec：scalable feature learning for networks[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2016：855-864.
[7] TANG J，QU M，WANG M，et al.LINE：large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web，2015：1067-1077.
[8] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[J].arXiv：1609.02907，2016.
[9] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[J].arXiv：1706.03762，2017.
[10] CHU X，YANG W，OUYANG W，et al.Multi-context attention for human pose estimation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1831-1840.
[11] WANG F，JIANG M，QIAN C，et al.Residual attention network for image classification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：3156-3164.
[12] ROY A G，NAVAB N，WACHINGER C.Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks[C]//International Conference on Medi-cal Image Computing and Computer-Assisted Intervention，2018：421-429.
[13] VELI?KOVI? P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[14] HAMILTON W L，YING R，LESKOVEC J.Inductive representation learning on large graphs[J].arXiv：1706. 02216，2017.
[15] GAO H，HUANG H.Deep attributed network embedding[C]//Twenty-Seventh International Joint Conference on Artificial Intelligence（IJCAI），2018.
[16] ZHANG Z，YANG H，BU J，et al.ANRL：attributed network representation learning via deep neural networks[C]//IJCAI，2018：3155-3161.
[17] SALEHI A，DAVULCU H.Graph attention auto-encoders[J].arXiv：1905.10715，2019.
[18] BELGHAZI M I，BARATIN A，RAJESHWAR S，et al.Mutual information neural estimation[C]//International Conference on Machine Learning，2018：531-540.
[19] HJELM R D，FEDOROV A，LAVOIE-MARCHILDON S，et al.Learning deep representations by mutual information estimation and maximization[J].arXiv：1808.06670，2018.
[20] VELI?KOVI? P，FEDUS W，HAMILTON W L，et al.Deep graph infomax[C]//International Conference on Learning Representations，2018.
[21] PENG Z，HUANG W，LUO M，et al.Graph representation learning via graphical mutual information maximization[C]//Proceedings of the Web Conference，2020.
[22] CHEN L，LE W，HONG R，et al.Revisiting graph based collaborative filtering：a linear residual graph convolutional network approach[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：27-34.
[23] HE X，DENG K，WANG X，et al.LightGCN：simplifying and powering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval，2020：639-648.
[24] SEN P，NAMATA G，BILGIC M，et al.Collective classification in network data[J].AI Magazine，2008，29（3）：93.
[25] PASZKE A，GROSS S，CHINTALA S，et al.Automatic differentiation in pytorch[C]//31st Conference on Neural Information Processing Systems（NIPS 2017），2017.
[26] GLOROT X，BENGIO Y.Understanding the difficulty of training deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics，2010：249-256.
[27] KINGMA D P，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.
[28] XU B，SHEN H，CAO Q，et al.Graph wavelet neural network[J].arXiv：1904.07785，2019.
[29] ZHU X，GHAHRAMANI Z，LAFFERTY J D.Semi-supervised learning using Gaussian fields and harmonic functions[C]//Proceedings of the 20th International Conference on Machine Learning（ICML-03），2003：912-919.
[30] KIPF T N，WELLING M.Variational graph auto-encoders[J].arXiv：1611.07308，2016.
[31] YANG Z，COHEN W，SALAKHUDINOV R.Revisiting semi-supervised learning with graph embeddings[C]//International Conference on Machine Learning，2016：40-48.
[32] VAN DER MAATEN L，HINTON G.Visualizing data using t-SNE[J].Journal of Machine Learning Research，2008，9（86）：2579-2605.