Deep Neural Network Combined with Graph Convolution for Text Classification

doi:10.3778/j.issn.1002-8331.2010-0172

Abstract

Abstract: With the development of graph convolutional network, graph convolutional network has been applied to many tasks, including text classification. By representing the text data as graph data, and then applying graph convolution on the graph, the structural information of the text and the long-distance dependence between words are captured, and good classification results are obtained. However, after the text is modeled as a graph model, the graph convolutional network faces the problem that the semantic information and local information of the text context are not fully expressed. A new model is proposed, which uses bi-directional long-short-term memory network（Bi_LSTM） and convolutional neural network（CNN） to extract the context semantic information and local feature information of text to enrich the text representation of graph convolutional network（GCN）, thus making up for the deficiency of graph convolutional network. At the same time, the graph pooling layer is used to filter out important nodes to help convolutional neural network capture the deep local feature information of text, which makes the model better represent text information. The experimental results on three English datasets show that the model has better classification effect than the baseline model.

Key words: text classification, neural network, graph convolutional network（GCN）

摘要： 随着图卷积网络的发展，图卷积网络已经应用到很多任务中，其中就包含文本分类任务。通过将文本数据表示成图数据，进而在图上应用图卷积，从而捕获文本的结构信息和单词间的长距离依赖关系获得了良好的分类效果。但将文本建模成图模型后，图卷积网络面临着文本上下文语义信息和局部特征信息表示不充分的问题。提出一种新的模型，利用双向长短时记忆网络（Bi_LSTM）和卷积神经网络（CNN）混合提取文本的上下文语义信息和局部特征信息去丰富图卷积网络（GCN）的文本表示，从而弥补图卷积网络的不足，同时使用图池化层筛选出重要节点帮助卷积神经网络捕获文本深层局部特征信息，使得模型能更好的表示文本信息。通过在3个英文数据集上的实验结果表明，该模型相比于基线模型有较好的分类效果。

关键词: 文本分类, 神经网络, 图卷积网络（GCN）

ZHENG Cheng, CHEN Jie, DONG Chunyang. Deep Neural Network Combined with Graph Convolution for Text Classification[J]. Computer Engineering and Applications, 2022, 58(7): 206-212.

郑诚, 陈杰, 董春阳. 结合图卷积的深层神经网络用于文本分类[J]. 计算机工程与应用, 2022, 58(7): 206-212.

References

[1] HARRIS Z.Distributional structure[J].Word，1954，10：2-3.
[2] ROUSSEAU F，KIAGIAS E，VAZIRGIANNIS M.Text categorization as a graph classification problem[C]//Proceedings of ACL 2015，2015：1702-1712.
[3] LUO U，SZOLOVITS B.Semantics and syntax with graph algorithms—state-of-the-art of extracting biomedical relations[J].Briefings in Bioinformatics，2016，18（1）：160-178.
[4] MIKOLOV T.Distributed representations of words and phrases and their compositionality[J].Advances in Neural Information Processing Systems，2013，26：3111-3119.
[5] JOULIN A，GRAVE E，BOJANOWSKI P，et al.Bag of tricks for efficient text classification[C]//Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics，2017：427-431.
[6] KIM Y.Convolutional neural networks for sentence classification[J].arXiv：1408.5882，2014.
[7] LIU P，QIU X，HUANG X.Recurrent neural network for text classification with multi-task learning[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence，2016.
[8] WANG G，LI C，WANG W，et al.Joint embedding of words and labels for text classification[C]//Proceedings of ACL 2018，2018：2321-2331.
[9] KIPF T N，WELLING M.Semi-supervised classificationwith graph convolutional networks[C]//Proceedings of the International Conference on Learning Representations，2017.
[10] VELICKOVIC P，CUCURULL G，CASANOVA A，et al.Graph attention networks[C]//Proceedings of the International Conference on Learning Representations，2018.
[11] XU K，BA J，KIROS R，et al.Show，attend and tell：Neural image caption generation with visual attention[C]//Proceedings of the International Conference on Machine Learning，2015.
[12] YAO L，MAO C，LUO Y.Graph convolutional networks for text classification[C]//Proceedings of AAAI 2019，2019.
[13] PENNINGTON J，SOCHER R，MANNING C D.Glove：Global vectors for word representation[C]//Proceedings of EMNLP 2014，2014.
[14] ZENG D J，LIU K，LAI S W，et al.Relation classification via convolutional deep neural network[C]//Proceedings of COLING 2014，2014.
[15] NAIR V，HINTON G E.Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the International Conference on Machine Learning，2010.
[16] HAN J，MORAGA C.The influence of the sigmoid function parameters on the speed of backpropagation learning[C]//Proceedings of International Workshop on Artificial Neural Networks：from Natural to Artificial Neural Computation，1995.
[17] ZHANG X，ZHAO J，LECUN Y.Character-level convolutional networks for text classification[J].arXiv：1509. 01626，2015.
[18] PANG B，LEE L.Seeing stars：Exploiting class relationships for sentiment categorization with respect to rating scales[C]//Proceedings of the 43rd Meeting on Association for Computational Linguistics，2005.
[19] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[20] TANG J，QU M，MEI Q Z.Pte：Predictive text embedding through large-scale heterogeneous text networks[C]//Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2015：1165-1174.
[21] KINGMA D，BA J.Adam：A method for stochastic optimization[C]//Proceedings of the International Conference on Learning Representations，2015.