Abstract:

Due to the introduction of the Bidirectional Encoder Representations from Transformers（BERT）, the traditional neural network method for solving sentence-level text sentiment analysis problems has been changed. The current deep learning model BERT’s own learning mode is unsupervised learning, which needs to rely on subsequent tasks to complete the reasoning and decision-making links, so there is a problem of lack of target domain knowledge. This paper proposes a Multi-level collaborative Convolutional Neural Network（MCNN） model, which can learn different levels of emotional features to supplement domain knowledge, and uses the BERT pre-training model to provide word vectors, and the sentence is dynamically adjusted through the dynamic adjustment of the BERT learning ability. The real sentiment orientation is embedded in the model, and finally the feature information output by the different levels of the model is combined with the output information of the bidirectional long-term short-term memory network to calculate the final text sentiment orientation. Experimental results show that even in corpora of different languages, this model has significantly improved the ability of emotional polarity classification compared with traditional neural networks and the recently proposed BERT-based deep learning model.

Key words: deep learning, text sentiment analysis, Bidirectional Encoder Representations from Transformers（BERT）, Convolutional Neural Network（CNN）, collaborative structure

摘要：

由于基于变换器的双向编码器表征技术（Bidirectional Encoder Representations from Transformers，BERT）的提出，改变了传统神经网络解决句子级文本情感分析问题的方法。目前的深度学习模型BERT本身学习模式为无监督学习，其需要依赖后续任务补全推理和决策环节，故存在缺乏目标领域知识的问题。提出一种多层协同卷积神经网络模型（Multi-level Convolutional Neural Network，MCNN），该模型能学习到不同层次的情感特征来补充领域知识，并且使用BERT预训练模型提供词向量，通过BERT学习能力的动态调整将句子真实的情感倾向嵌入模型，最后将不同层次模型输出的特征信息同双向长短期记忆网络输出信息进行特征融合后计算出最终的文本情感性向。实验结果表明即使在不同语种的语料中，该模型对比传统神经网络和近期提出的基于BERT深度学习的模型，情感极性分类的能力有明显提升。

关键词: 深度学习, 文本情感分析, 基于变换器的双向编码器表征技术（BERT）, 卷积神经网络（CNN）, 协同结构