Dual-Channel Sentiment Analysis and Application Based on Gated Attention

doi:10.3778/j.issn.1002-8331.2112-0585

Abstract

Abstract: Traditional deep learning-based text sentiment classification models usually cannot extract features completely and cannot distinguish polysemous words. To resolve these problems, a dual-channel sentiment classification model named BGA-DNet based on gated attention is proposed. The model uses the BERT pre-training model to process text data, and then extracts text features through a dual-channel network. The channel one uses TextCNN to extract local features, and the channel two uses BiLSTM-Attention to extract global features. At the same time, a gated attention unit is introduced to filter out some useless attention information. With a residual network, it also ensures that the output of the dual-channel retains the original coding information when the network reaches a saturated state. BGA-DNet is evaluated on two public datasets of hotel reviews and restaurant reviews, and compared with the latest sentiment classification methods, it achieves the best results with accuracy rate of 94.09% and 91.82%, respectively. At last, the BGA-DNet model is applied to the real dataset of students’ experiment reports, and the accuracy and [F1] value are also the highest.

Key words: gated attention, dual-channel, sentiment classification, BERT, BiLSTM-Attention

摘要： 针对传统的基于深度学习的文本情感分类模型特征抽取不全面以及不能区分一词多义的问题，提出一种基于门控注意力的双通道情感分类模型BGA-DNet。该模型使用BERT预训练模型对文本数据进行处理，然后经过双通道网络提取文本特征，其中通道一利用TextCNN提取局部特征，通道二利用BiLSTM-Attention提取全局特征。同时引入门控注意力单元将部分无用的注意力信息过滤掉，并结合残差网络思想，确保双通道的输出在网络学习到饱和状态下保留原始编码信息。BGA-DNet在公开的酒店评论和餐饮评论两个数据集上进行实验评估，并与最新的情感分类方法进行对比，分别取得了准确率94.09%和91.82%的最佳效果。最后将BGA-DNet模型应用到真实的学生实验心得体会评价任务上，与其他方法相比准确率和[F1]值也是最高的。

关键词: 门控注意力, 双通道, 情感分类, BERT, BiLSTM-Attention

WEI Long, HU Jianpeng, ZHANG Geng. Dual-Channel Sentiment Analysis and Application Based on Gated Attention[J]. Computer Engineering and Applications, 2023, 59(10): 134-141.

魏龙, 胡建鹏, 张庚. 基于门控注意力的双通道情感分析及应用[J]. 计算机工程与应用, 2023, 59(10): 134-141.

References

[1] 王婷，杨文忠.文本情感分析方法研究综述[J].计算机工程与应用，2021，57（12）：11-24.
WANG T，YANG W Z.Review of text sentiment analysis methods[J].Computer Engineering and Applications，2021，57（12）：11-24.
[2] VAPNIK V N，CHERVONENKIS A.A note on one class of perceptron[J].Automation and Remote Control，1964，25（1）.
[3] ALTMAN N S.An introduction to kernel and nearest neighbor nonparametric regression[J].The American Statistician，1992，46（3）：175-185.
[4] 徐建忠，朱俊，赵瑞，等.基于SVM算法的航天微博情感分析[J].信息安全研究，2017，3（12）：1129-1133.
XU J Z，ZHU J，ZHAO R，et al.Sentiment analysis of aerospace microblog using SVM[J].Journal of Information Security Research，2017，3（12）：1129-1133.
[5] 梁杰，陈嘉豪，张雪芹，等.基于独热编码和卷积神经网络的异常检测[J].清华大学学报（自然科学版），2019，59（7）：523-529.
LIANG J，CHEN J H，ZHANG X Q，et al.One-hot encoding and convolutional neural network based anomaly detection[J].Journal of Tsinghua University（Science and Technology），2019，59（7）：523-529.
[6] JONES K S.IDF term weighting and IR research lessons[J].Journal of Documentation，2004，60（5）：521-523.
[7] YANG X，XU S，WU H，et al.Sentiment analysis of Weibo comment texts based on extended vocabulary and convolutional neural network[J].Procedia Computer Science，2019，147：361-368.
[8] JELODAR H，WANG Y，ORJI R，et al.Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions：NLP using LSTM recurrent neural network approach[J].IEEE Journal of Biomedical and Health Informatics，2020，24（10）：2733-2742.
[9] 彭祝亮，刘博文，范程岸，等.基于BLSTM与方面注意力模块的情感分类方法[J].计算机工程，2020，46（3）：60-65.
PENG Z L，LIU B W，FAN C A，et al.Sentiment classification method based on BLSTM and aspect attention module[J].Computer Engineering，2020，46（3）：60-65.
[10] BAHDANAU D，CHO K，BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv：1409.0473，2014.
[11] LI M Z，CHEN L，ZHAO J，et al.Sentiment analysis of Chinese stock reviews based on BERT model[J].Applied Intelligence，2021，51：5016-5024.
[12] 杨长利，刘智，鲁明羽.双通道混合神经网络的文本情感分析模型[J].计算机工程与应用，2020，56（11）：124-128.
YANG C L，LIU Z，LU M Y.Text sentiment analysis model of two-channel hybrid neural network[J].Computer Engineering and Applications，2020，56（11）：124-128.
[13] GIMNEZ M，PALANCA J，BOTTI V.Semantic-based padding in convolutional neural networks for improving the performance in natural language processing.A case of study in sentiment analysis[J].Neurocomputing，2020，378：315-323.
[14] 方炯焜，陈平华，廖文雄.结合GloVe和GRU的文本分类模型[J].计算机工程与应用，2020，56（20）：98-103.
FANG J K，CHEN P H，LIAO W X.Text classification model based on GloVe and GRU[J].Computer Engineering and Applications，2020，56（20）：98-103.
[15] ZADEH A，LIANG P P，MAZUMDER N，et al.Memory fusion network for multi-view sequential learning[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence.Palo Alto：AAAI，2018：5634-5641.
[16] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems 30，2017：5998-6008.
[17] DEVLIN J，CHANG M W，LEE K，et al.BERT：pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2019：4171-4186.
[18] MISRA D.Mish：a self regularized non-monotonic activation function[J].arXiv：1908.08681，2019.
[19] YU C，SU Z.Symmetrical Gaussian error linear units （SGELUs）[J].arXiv：1911.03925，2019.
[20] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，2014：1746-1751.
[21] LAI S，XU L.Recurrent convolutional neural networks for text classification[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence，2015：2267-2273.
[22] 林渤，薛斌，林逸滔，等.面向初学者的C语言在线实验系统设计与实现[J].智能计算机与应用，2020，10（10）：108-111.
LIN B，XUE B，LIN Y T，et al.Design and implementation of C language online experiment system for beginners[J].Intelligent Computer and Applications，2020，10（10）：108-111.