Aspect-Level Sentiment Analysis Based on BG-DATT-CNN Network

doi:10.3778/j.issn.1002-8331.2105-0483

Abstract

Abstract: This paper aims to analyze the emotional polarity of sentences towards different aspects, mine the information in the commentary text deeply, and provide suggestions for enterprises to make production decisions. Traditional methods mostly consider the single level of attention information and the RNN-based model ignores the importance of local features while the CNN-based model cannot capture long-distance dependent information. To solve the above problems, a model named BG-DATT-CNN based on dual attention mechanism is proposed. In terms of feature representation, BERT is used to encode the sentence and aspect respectively, so as to obtain the deep semantic features of the text. In terms of feature extraction, a dual attention mechanism is designed to obtain the comprehensive weight by calculating two kinds of weights to strengthen the context-related features and aspect-related features of sentences. In terms of model design, the BG-DATT-CNN network combines the advantages of GRU and CNN. The Bi-GRU layer captures the global features of context, and the CNN layer, which includes the [K]-Max pooling layer and TextCNN layer, obtains the key information of classification through two-stage feature extraction. Experiments on the SemEval 2014 dataset show that compared with other existing models, the proposed model achieves better results.

Key words: aspect-level sentiment analysis（ALSA）, attention machanism, Bi-GRU, [K]-Max pooling, TextCNN

摘要： 分析句子针对不同方面的情感极性，深入挖掘评论文本中的信息，为企业生产决策提供建议。针对传统方法多考虑单一层面注意力信息，且基于RNN的模型忽略了局部特征的重要性，而基于CNN的模型不能捕捉长距离依赖的信息的问题，提出了基于双重注意力机制的BG-DATT-CNN模型。在特征表示上，利用BERT对句子和方面词分别进行词向量编码，获得文本的深层语义特征。在特征提取上，设计了双重注意力机制，通过计算两类权重获得综合权重，强化文本的上下文相关特征和方面相关特征。在模型构建上，设计了BG-DATT-CNN网络，结合GRU和CNN各自的优势，Bi-GRU层捕捉文本的上下文全局特征，CNN层包括[K]-Max池化层和TextCNN层，通过两阶段特征提取获取分类的关键信息。在SemEval 2014数据集上的实验表明，与现有的其他模型相比，提出的模型取得了较好的效果。

关键词: 方面级别情感分类, 双重注意力机制, Bi-GRU, [K]-Max池化, TextCNN

YU Bengong, WANG Huiling, ZHU Xiaojie. Aspect-Level Sentiment Analysis Based on BG-DATT-CNN Network[J]. Computer Engineering and Applications, 2022, 58(24): 151-157.

余本功, 王惠灵, 朱晓洁. 基于BG-DATT-CNN网络的方面级别情感分析[J]. 计算机工程与应用, 2022, 58(24): 151-157.

References

[1] MANEK A S，SHENOY P D，MOHAN M C，et al.Aspect term extraction for sentiment analysis in large movie reviews using Gini Index feature selection method and SVM classifier[J].World Wide Web，2017，20（2）：135-154.
[2] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[3] TANG D，QIN B，LIU T.Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，2016：214-224.
[4] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017：5998-6008.
[5] KAJI N，KITSUREGAWA M.Building lexicon for sentiment analysis from massive collection of HTML documents[C]//Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning（EMNLP-CoNLL），2007：1075-1083.
[6] GURKHE D，PAL N，BHATIA R.Effective sentiment analysis of social media datasets using Naive Bayesian classification[J].International Journal of Computer Applications，2014，975（8887）：99.
[7] QIAN Q，TIAN B，HUANG M，et al.Learning tag embeddings and tag-specific composition functions in recursive neural network[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing（Volume 1：Long Papers），2015：1365-1374.
[8] SOCHER R，PENNINGTON J，HUANG E H，et al.Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing，2011：151-161.
[9] TAI K S，SOCHER R，MANNING C D.Improved semantic representations from tree-structured long short-term memory networks[C]//Proceedings of ACL，2015.
[10] JIANG L，YU M，ZHOU M，et al.Target-dependent twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics：Human Language Technologies，2011：151-160.
[11] TANG D，QIN B，FENG X，et al.Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of COLING 2016，the 26th International Conference on Computational Linguistics：Technical Papers，2016：3298-3307.
[12] KUMAR A，IRSOY O，ONDRUSKA P，et al.Ask me anything：dynamic memory networks for natural language processing[C]//International Conference on Machine Learning，2016：1378-1387.
[13] LIN Y，SHEN S，LIU Z，et al.Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2016：2124-2133.
[14] CUI Y，CHEN Z，WEI S，et al.Attention-over-attention neural networks for reading comprehension[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2017：593-602.
[15] WANG Y，HUANG M，ZHU X，et al.Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，2016：606-615.
[16] TAY Y，TUAN L A，HUI S C.Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[17] MA D，LI S，ZHANG X，et al.Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence，2017：4068-4074.
[18] HUANG B，OU Y，CARLEY K M.Aspect level sentiment classification with attention-over-attention neural networks[C]//International Conference on Social Computing，Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation.Cham：Springer，2018：197-206.
[19] FAN F，FENG Y，ZHAO D.Multi-grained attention network for aspect-level sentiment classification[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：3433-3442.
[20] 孙小婉，王英，王鑫，等.面向双注意力网络的特定方面情感分析模型[J].计算机研究与发展，2019，56（11）：2384-2395.
SUN X W，WANG Y，WANG X，et al.Aspect-based sentiment analysis model based on dual-attention networks[J].Journal of Computer Research and Development，2019，56（11）：2384-2395.
[21] XUE W，LI T.Aspect based sentiment analysis with gated convolutional networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2018：2514-2523.
[22] 梁斌，刘全，徐进，等.基于多注意力卷积神经网络的特定目标情感分析[J].计算机研究与发展，2017，54（8）：1724-1735.
LIANG B，LIU Q，XU J，et al.Aspect-based sentiment analysis based on multi-attention CNN[J].Journal of Computer Research and Development，2017，54（8）：1724-1735.
[23] DEVLIN J，CHANG M W，LEE K，et al.Bert：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[24] YANG Z，YANG D，DYER C，et al.Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2016：1480-1489.
[25] KIM Y.Convolutional neural networks for sentence classification[J].arXiv：1408.5882，2014.