中文短文本情感分类：融入位置感知强化的Transformer-TextCNN模型研究

doi:10.3778/j.issn.1002-8331.2403-0082

摘要/Abstract

摘要： 针对当前中文短文本情感分类模型文本位置信息与关键特征获取不足的问题，提出了一种融入位置感知强化的Transformer-TextCNN情感分类模型。利用BERT可学习绝对位置编码与正弦位置编码强化模型的位置感知能力，融合Transformer的全局上下文理解能力与TextCNN的局部特征捕捉能力，分别提取中文短文本全局特征与局部特征，构建位置感知强化与特征协同的情感特征输出服务，实现中文短文本情感准确分类。实验结果表明，该模型在视频弹幕数据集上的准确率达到90.23%，在SMP2020数据集上的准确率达到87.38%。相较于最优的基线模型，准确率在视频弹幕数据集和SMP2020数据集上分别提高了1.98和0.44个百分点，在中文短文本情感分类任务中取得更好的分类效果。

关键词: 文本情感分类, BERT, Transformer, textCNN, 位置编码

Abstract: Aiming at the problem of insufficient acquisition of text location information and key features in the current Chinese short text sentiment classification model, a Transformer-TextCNN sentiment classification model integrated with location-aware enhancement is proposed. Firstly, BERT’s learnable absolute position encoding and sinusoidal position coding are used to enhance the position perception ability of the model. Then, the global context understanding ability of Transformer and the local feature capture ability of TextCNN are combined to extract the global and local features of Chinese short text respectively. Finally, the emotional feature output service of position perception reinforcement and feature collaboration is constructed to realize the accurate classification of Chinese short text emotion. The experimental results show that the accuracy of the model on the video barrage data set reaches 90.23%, and the accuracy on the SMP2020 data set reaches 87.38%. Compared with the optimal baseline model, the accuracy rate is increased by 1.98 and 0.44 percentage points on the video barrage dataset and SMP2020 dataset, respectively, and better classification results are achieved in the Chinese short text sentiment classification task.

Key words: text sentiment classification, BERT, Transformer, textCNN, positional encoding

李浩君, 王耀东, 汪旭辉. 中文短文本情感分类：融入位置感知强化的Transformer-TextCNN模型研究[J]. 计算机工程与应用, 2025, 61(11): 216-226.

LI Haojun, WANG Yaodong, WANG Xuhui. Chinese Short Text Sentiment Classification: Research on Transformer-TextCNN Model with Location-Aware Enhancement[J]. Computer Engineering and Applications, 2025, 61(11): 216-226.

参考文献

[1] 刘志明, 刘鲁. 基于机器学习的中文微博情感分类实证研究[J]. 计算机工程与应用, 2012, 48(1): 1-4.
LIU Z M, LIU L. Empirical study of sentiment classification for Chinese microblog based on machine learning[J]. Computer Engineering and Applications, 2012, 48(1): 1-4.
[2] 徐军, 丁宇新, 王晓龙. 使用机器学习方法进行新闻的情感自动分类[J]. 中文信息学报, 2007(6): 95-100.
XU J, DING Y X, WANG X L. Sentiment classification for Chinese news using machine learning methods[J]. Journal of Chinese Information Processing, 2007(6): 95-100.
[3] 欧阳纯萍, 阳小华, 雷龙艳, 等. 多策略中文微博细粒度情绪分析研究[J]. 北京大学学报(自然科学版), 2014, 50(1): 67-72.
OUYANG C P, YANG X H, LEI L Y, et al. Multi-strategy approach for fine-grained sentiment analysisof Chinese microblog[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2014, 50(1): 67-72.
[4] 杨爽, 陈芬. 基于SVM多特征融合的微博情感多级分类研究[J]. 数据分析与知识发现, 2017, 1(2): 73-79.
YANG S, CHEN F. Analyzing sentiments of micro-blog posts based on support vector machine[J]. Data Analysis and Knowledge Discovery, 2017, 1(2): 73-79.
[5] 杨丽, 吴雨茜, 王俊丽, 等. 循环神经网络研究综述[J]. 计算机应用, 2018, 38(2): 1-6.
YANG L, WU Y X, WANG J L, et al. Research on recurrent neural network[J]. Journal of Computer Applications, 2018, 38(2): 1-6.
[6] 陈珂, 梁斌, 柯文德, 等. 基于多通道卷积神经网络的中文微博情感分析[J]. 计算机研究与发展, 2018, 55(5): 945-957.
CHEN K, LIANG B, KE W D, et al. Chinese micro-blog sentiment analysis based on multi-channels convolutional neural networks[J]. Journal of Computer Research and Development, 2018, 55(5): 945-957.
[7] 杨秀璋, 武帅, 张苗, 等. 基于TextCNN和Attention的微博舆情事件情感分析[J]. 信息技术与信息化, 2021(7): 41-46.
YANG X Z, WU S, ZHANG M, et al. Emotional analysis of microblog public opinion events based on TextCNN and Attention[J]. Information Technology and Informatization, 2021(7): 41-46.
[8] 李浩君, 吕韵, 汪旭辉, 等. 融入情感分析的多层交互深度推荐模型研究[J]. 数据分析与知识发现, 2023, 7(3): 43-57.
LI H J, LYU Y, WANG X H, et al. A deep recommendation model with multi-layer interaction and sentiment analysis[J]. Data Analysis and Knowledge Discovery, 2023, 7(3): 43-57.
[9] 李洋, 董红斌. 基于CNN和BiLSTM网络特征融合的文本情感分析[J]. 计算机应用, 2018, 38(11): 3075-3080.
LI Y, DONG H B. Text sentiment analysis based on feature fusion of convolution neural network and bidirectional long short-term memory network[J]. Journal of Computer Applications, 2018, 38 (11): 3075-3080.
[10] 王丽亚, 刘昌辉, 蔡敦波, 等. CNN-BiGRU网络中引入注意力机制的中文文本情感分析[J]. 计算机应用, 2019, 39(10): 2841-2846.
WANG L Y, LIU C H, CAI D B, et al. Chinese text sentiment analysis based on CNN-BiGRU network with attention mechanism[J]. Journal of Computer Applications, 2019, 39(10): 2841-2846.
[11] YAN C, LIU J, LIU W, et al. Sentiment analysis and topic mining using a novel deep attention-based parallel dual-channel model for online course reviews[J]. Cognitive Computation, 2023, 15(1): 304-322.
[12] 臧洁, 鲁锦涛, 王妍, 等. 融合双通道特征的中文短文本情感分类模型[J]. 计算机工程与应用, 2024, 60(21): 116-126.
ZANG J, LU J T, WAMG Y, et al. Chinese short text sentiment classification model integrating dual-channel features[J]. Computer Engineering and Applications, 2024, 60(21): 116-126.
[13] 陈俊涛, 刘力铭, 车月琴. 基于图卷积网络的中文短文本细粒度情感分析[J]. 电脑与电信, 2023, (3): 79-84.
CHEN J T, LIU L M, CHE Y Q. Sentiment analysis of fine-grained chinese short text based on graph convolution network[J]. Computer and Telecommunication, 2023, (3): 79-84.
[14] ZHANG Y, XU H, XU K. Chinese short text classification based on dependency syntax information[C]//Proceedings of the 5th International Conference on Compute and Data Analysis, 2021: 133-138.
[15] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[16] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st Conference on Neural Information Processing Systems, 2017: 6000-6010.
[17] SHAW P, USZKOREIT J, VASWANI A. Self-attention with relative position representations[J]. arXiv:1803.02155, 2018.
[18] 霍帅, 庞春江. 基于Transformer和多通道卷积神经网络的情感分析研究[J]. 计算机科学, 2021, 48(1): 349-356.
HUO S, PANG C J. Research on sentiment analysis based on Transformer and multi-channel convolutional neural network[J]. Computer Science, 2021, 48 (1): 349-356.
[19] 王金政, 杨颖, 余本功. 基于多头协同注意力机制的客户投诉文本分类模型[J]. 数据分析与知识发现, 2023, 7(1): 128-137.
WANG J Z, YANG Y, YU B G. Classifying customer complaints based on multi-head co-attention mechanism[J]. Data Analysis and Knowledge Discovery, 2023, 7(1): 128-137.
[20] KIM Y. Convolutional neural networks for sentence classification[J]. arXiv:1408.5882, 2014.
[21] 申昌, 冀俊忠. 基于双通道卷积神经网络的文本情感分类算法[J]. 模式识别与人工智能, 2018, 31(2): 158-166.
SHEN C, JI J Z. Text sentiment classification algorithm based on double channel convolutional neural network[J]. Pattern Recognition and Artificial Intelligence, 2018, 31 (2): 158-166.
[22] 杨奎河, 刘智鹏. 基于BERT-BiLSTM的短文本情感分析[J]. 长江信息通信, 2020(6): 81-82.
YANG K H, LIU Z P. Short text sentiment analysis based on BERT-BiLSTM[J]. Changjiang Information and Communications, 2020(6): 81-82.
[23] 邵辉. 基于BERT-TextCNN的中文短文本情感分析[J]. 信息与电脑, 2022, 34(1): 77-80.
SHAO H. Sentiment analysis of Chinese short text based on BERT-TextCNN[J]. Information and Computer, 2022, 34(1): 77-80.
[24] 张浩然, 谢云熙, 张艳荣. 基于TextCNN的文本情感分类系统[J]. 哈尔滨商业大学学报(自然科学版), 2022, 38(3): 285-292.
ZHANG H R, XIE Y X, ZHANG Y R. Text sentiment classification system based on TextCNN[J]. Journal of Harbin University of Commerce (Natural Science Edition), 2022, 38(3): 285-292.
[25] 和志强, 杨建, 罗长玲. 基于BiLSTM神经网络的特征融合短文本分类算法[J]. 智能计算机与应用, 2019, 9(2): 21-27.
HE Z Q, YANG J, LUO C L. Combination characteristics based on BiLSTM for short text classification[J]. Intelligent Computer and Applications, 2019, 9(2): 21-27.
[26] 诸林云, 曲金帅, 范菁, 等. 基于BERT-BiLSTM-Attention的文本情感分析[J]. 云南民族大学学报(自然科学版), 2023, 32(4): 520-527.
ZHU L Y, QU J S, FAN J, et al. Text sentiment analysis based on BERT-BiLSTM-Attention [J]. Journal of Yunnan Minzu University(Natural Sciences Edition), 2023, 32(4): 520-527.