Improved Graph Attention Mechanism Model for Graph Embedding

doi:10.3778/j.issn.1002-8331.2205-0432

Abstract

Abstract: Addressing the problem of easy loss of large amounts of feature information and the incompletely preserved graph topology in the process of representation of embedded nodes in learning graph in the graph neural network model, an improved graph attention mechanism model is proposed. The model is divided into the node-level bidirectional attention mechanism and the graph-level self-attention graph pooling. During the representation of new feature vectors of the learning graph nodes, calculations of the bidirectional graph attention weight are adopted to provide reliable choices for the retention of neighboring nodes while enhancing similar properties between nodes. A self-attention graph pooling is combined with the overall topology of the graph, using node feature vectors as input to generate a graph embedding representation at the pooling layer through the self-attention weights provided by the attention convolution layer. It is tested on Cora, Citeseer, and Pubmed datasets. According to the experimental results, compared with the baseline graph attention mechanism model, the improved model takes full account of the local and global structural features of the graph. It effectively enhances the ability of the model to aggregate neighborhood information, reduces the loss of original features in graph embedding, and significantly improves the performance of the model for downstream tasks.

Key words: graph embedding, bidirectional graph attention mechanism, self-attention graph pool, feature representation, graph topology

摘要： 针对图神经网络模型学习图嵌入节点表示过程中易丢失大量特征信息及其图拓扑保留不完整的问题，提出了一种改进的图注意力（graph attention）机制模型。该模型分为节点级双向注意力机制和图级自注意图池化两部分。在学习图节点新的特征向量表示过程中，采取计算双向图注意力权重的方式，为邻域节点的保留提供可靠选择的同时增强节点间的相似属性；在图的整体拓扑上结合自注意图池，使用节点特征向量作为输入，通过注意卷积层提供的自注意权重在池化层生成图嵌入表示；在Cora、Citeseer、Pubmed数据集上进行了测试，实验结果表明：相比于基准图注意力机制模型，改进模型能够充分考虑图的局部和整体结构特征，有效增强模型聚合邻域信息的能力，减少了图嵌入过程中原始特征的丢失，明显提升了模型在下游任务的表现性能。

关键词: 图嵌入, 双向图注意力机制, 自注意图池, 特征表示, 图拓扑

LI Zhijie, HAN Jinjin, LI Changhua, ZHANG Jie. Improved Graph Attention Mechanism Model for Graph Embedding[J]. Computer Engineering and Applications, 2023, 59(17): 152-158.

李智杰, 韩津津, 李昌华, 张颉. 面向图嵌入的改进图注意机制模型[J]. 计算机工程与应用, 2023, 59(17): 152-158.

References

[1] BIANCHI F M，GRATTAROLA D，LIVI L，et al.Graph neural networks with convolutional aRMA filters[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，44（7）：3496-3507.
[2] HENRY N，FEKETE J D，MCGUFFIN M.NodeTrix：a hybrid visualization of social networks[J].IEEE Trans Vis Comput Graph，2007，13（6）：1302-1309.
[3] CHEN M，WEI Z，HUANG Z，et al.Simple and deep graph convolutional networks[J].arXiv：2007.02133，2020.
[4] CHAMI I，YING R，Ré C，et al.Hyperbolic graph convolutional neural networks[C]//Advances in Neural Information Processing Systems，2019：4869-4880.
[5] LIU F，XUE S，WU J，et al.Deep learning for community detection：progress，challenges and opportunities[J].arXiv：2005.08225，2020.
[6] 邹文武.社交网络用户影响力的发展动态及知识图谱研究[J].情报科学，2020，38（9）：107-115.
ZOU W W.Research on the development trend and knowledge map of social network users’ influence[J].Information Science，2020，38（9）：107-115.
[7] 祁志卫，王笳辉，岳昆，等.图嵌入方法与应用：研究综述[J].电子学报，2020，48（4）：808-818.
QI Z W，WANG J H，YUE K，et al.Methods and applications of graph embedding：a survey[J].Acta Electronica Sinica，2020，48（4）：808-818.
[8] CAO S，LU W.Grarep：learning graph representations with global structural information[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management，New York，USA，2015：891-900.
[9] PEROZZI B，AL-RFOU R，SKIENA S.DeepWalk：online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining（KDD’14），New York，NY，USA，2014：701-710.
[10] WU Z，PAN S，CHEN F，et al.A comprehensive survey on graph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems，2021，32（1）：4-24.
[11] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[J].arXiv：1609.02907，2016.
[12] HAMILTON W L，YING R，LESKOVEC J.Inductive representation learning on large graphs[J].arXiv：1706. 02216，2017.
[13] VELIKOVI P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[14] XU K，LI C，TIAN Y，et al.Representation learning on graphs with jumping knowledge networks[J].arXiv：1806.03536，2018.
[15] MENG L，ZHANG J.IsoNN：isomorphic neural network for graph representation learning and classification[J].arXiv：1907.09495，2019.
[16] CA I H，ZHENG V W，CHANG C C.A comprehensive survey of graph embedding：problems，techniques and applications[J].IEEE Transactions on Knowledge and Data Engineering，2018，30（9）：1616-1637.
[17] GIBERT J，VALVENY E，BUNKE H.Graph embedding in vector spaces by node attributestatistics[J].Pattern Recognition，2012，45（9）：3072-3083.
[18] YAN S，XU D，ZHANG B，et al.Graph embedding and extensions：a general framework for dimensionality reduction[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2007，29（1）：40-51.
[19] 王佳.图神经网络浅析[J].现代计算机，2019（23）：58-62.
WANG J.Analysis of graph neural network[J].Modern Computer，2019（23）：58-62.
[20] BALDASSARRE F，AZIZPOUR H.Explainability techniques for graph convolutional networks[J].arXiv：1905. 13686，2019.
[21] LEE J，LEE I，KANG J.Self-attention graph pooling[J].arXiv：1904.08082，2019.
[22] HE T，BAI L，ONG Y S.Graph joint attention networks[J].arXiv：2102.03147，2021.
[23] ABU-EL-HAIJA S，PEROZZI B，AL-RFOU R，et al.Watch your step：learning node embeddings via graph attention[J].arXiv：1710.09599，2017.