关键词: 文本分类, 循环卷积神经网络, 池化, 最大池化, 多头注意力池化

Abstract:

The strategy of max pooling in the pooling layer adopted by classic Recurrent Convolutional Neural Network（RCNN） is relatively onefold, which will ignore other features except the most prominent one and affect the classification accuracy. Therefore, a Multi-Head Attention Pooling-based Recurrent Convolutional Neural Network（MHAP-RCNN） is proposed. The mechanism of multi-head attention pooling can fully consider the contribution of each feature to classification, and can be dynamically optimized in the training process, which can effectively alleviate the above problem of max pooling. Experiments are performed on three public text classification data sets. The results show that the proposed model has better performance on text classification than classic RCNN and other models.

Key words: text classification, recurrent convolutional neural network, pooling, max pooling, multi-head attention pooling