Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (17): 107-115.DOI: 10.3778/j.issn.1002-8331.2211-0397

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Text Classification Method Based on Integration of Bert and Hypergraph Convolutional Network

LI Quanxin, PANG Jun, ZHU Fengran   

  1. 1.School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430070, China
    2.Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System, Wuhan 430070, China
  • Online:2023-09-01 Published:2023-09-01

结合Bert与超图卷积网络的文本分类模型

李全鑫,庞俊,朱峰冉   

  1. 1.武汉科技大学 计算机科学与技术学院,武汉 430070
    2.智能信息处理与实时工业系统湖北省重点实验室,武汉 430070

Abstract: When the existing graph neural networks deal with the problem of text classification, the text is usually transformed into graph structure, and then the graph neural network is used to learn and obtain the final classifications. However, the methods based on graph neural networks have two shortcomings. One is that graph structure uses binary connection to represent word relations, and it lacks the ability of capturing higher-order relations in the text. The second is that it is difficult for the graph neural networks to capture the rich semantic associations. To solve the above problems, a text classification model is proposed to integrate Bert with hypergraph convolutional network using attention mechanism. Local semantic information in the text is obtained by using the Bert model. Text association information between words is obtained by constructing the text hypergraph, and the global structure features are obtained by learning of the hypergraph convolutional network. Attention mechanism is used to interact with the two features to obtain a more comprehensive and adequate text representation. Extensive experiments on the four public datasets show that the model outperforms the baseline models.

Key words: text classification, Bert, hypergraph, neural network

摘要: 现有的图神经网络在处理文本分类问题时,通常将文本转化为图结构,然后利用图神经网络进行学习表示,从而分类。但基于图神经网络的方法存在两点不足,一是图结构利用二元关系对单词联系进行表示,缺乏对文本高阶关系的捕获;二是图神经网络类模型难以捕获文本中丰富的语义关联。针对以上问题,提出了一种采用注意力机制将Bert与超图卷积网络结合的文本分类模型。通过Bert模型获得文本中的局部语义信息,通过构建文本超图获得单词间更广泛的文本关联信息,并经过超图卷积网络的学习表示得到文本全局结构特征,利用注意力机制对两种特征进行交互影响,获得更全面更充分的文本表示。在4个公开数据集上进行的多次实验表明,该模型与基线模型相比有更好的分类效果。

关键词: 文本分类, Bert, 超图, 神经网络