Abstract: It is difficult to extract enough text information from multi-classification tasks with bi-directional long short-term memory（BiLSTM） and convolutional neural network（CNN）. A BiLSTM_CNN compound model based on self-attention mechanism and residual network（ResNet） is proposed. The weight of the information after the convolution operation is given by the self-attention mechanism. The pooling feature information is processed by layer normalization and then connects to the residual network, so that the model can learn the residual informationand further improve the classification performance of the model. In the process of calculation, Mish nonlinear activation function is applied, which is more smooth than the common Relu function. Compared with common deep learning models, the proposed method is superior to the existing mainstream models in terms of the accuracy and F1 evaluation indicators. The proposed model provides new research ideas for text classification problems.

Key words: self-attention mechanism, bi-directional long short-term memory network（BiLSTM） , residual network, convolutional neural network（CNN）, layer normalization

摘要： 双向长短期记忆网络（BiLSTM）和卷积神经网络（CNN）很难在文本的多分类任务中提取到足够的文本信息。提出了一种基于自注意力机制（self_attention）和残差网络（ResNet）的BiLSTM_CNN复合模型。通过自注意力赋予卷积运算后信息的权重，接着将池化后的特征信息层归一化并接入残差网络，让模型学习到残差信息，从而进一步提高模型的分类性能。在模型的运算过程中，使用了更加光滑的Mish非线性激活函数代替Relu。通过与深度学习模型对比，所提出的方法在准确率以及F1值评价指标上均优于现有模型，为文本分类问题提供了新的研究思路。

关键词: 自注意力机制, 双向长短期记忆网络, 残差网络, 卷积神经网络, 层归一化