Aspect-Level Sentiment Analysis Based on Location-Enhanced Word Embeddings and GRU-CNN Model

doi:10.3778/j.issn.1002-8331.2212-0375

Abstract

Abstract: Aspect-level sentiment analysis aims to judge the emotional attitude of a specific aspect-level word according to the given context. The core problem is how to accurately represent the context of the aspect word. Different from the existing research which mainly focuses on the improvement of attention mechanism, this paper focuses on two aspects： word representation and context encoding. In terms of word representation, the location-enhanced word representation is obtained through the BERT model and position measurement formula. In terms of context encoding, GRU-CNN network is used to extract semantic features of text. The experiments on the SemEval2014 Task4 dataset show that the accuracy of the proposed model on Restaurant and Laptop datasets respectively reaches to 85.54% and 80.35%, which proves the efficiency of the proposed model.

Key words: aspect-level sentiment analysis, convolutional neural network (CNN), pre-trained word vectors, position function, attention mechanism

摘要： 方面级情感分析旨在判断一段文本中特定方面词的情感倾向，其核心问题是方面词的上下文如何准确表征。与现有研究主要关注注意力机制的改进不同，该文从词语表征和上下文编码模型两个方面进行改进。在词语表征方面，通过BERT模型和位置度量公式获得增强的词向量表示；在上下文编码模型方面，使用GRU-CNN网络提取文本语义特征。在SemEval2014 Task4数据集上的实验表明，提出的模型在Restaurant和Laptop领域中的准确率分别达到了85.54%和80.35%，证实了所提出模型的有效性。

关键词: 方面级情感分析, 卷积神经网络, 预训练词向量, 位置函数, 注意力机制

TAO Linjuan, HUA Gengxing, LI Bo. Aspect-Level Sentiment Analysis Based on Location-Enhanced Word Embeddings and GRU-CNN Model[J]. Computer Engineering and Applications, 2024, 60(9): 212-218.

陶林娟, 华庚兴, 李波. 基于位置增强词向量和GRU-CNN的方面级情感分析模型研究[J]. 计算机工程与应用, 2024, 60(9): 212-218.

References

[1] 冯超, 黎海辉, 赵洪雅, 等. 基于层次注意力机制和门机制的属性级别情感分析[J]. 中文信息学报, 2021, 35(10): 128-136.
FENG C, LI H H, ZHAO H Y, et al. Aspect-level sentiment analysis based on hierarchical attention and gate networks[J]. Journal of Chinese Information Processing, 2021, 35(10): 128-136.
[2] YANG C, ZHANG H, JIANG B, et al. Aspect-based sentiment analysis with alternating coattention networks[J]. Information Processing and Management, 2019, 56(3): 463-478.
[3] WANG Y, HUANG M, ZHU X, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016.
[4] SHUANG K, ZHANG Z, GUO H, et al. A sentiment information collector-extractor architecture based neural network for sentiment analysis[J]. Information Sciences, 2018, 467: 549-558.
[5] WANG L, NIU J, YU S. SentiDiff: combining textual information and sentiment diffusion patterns for Twitter sentiment analysis[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 32(10): 2026-2039.
[6] SERRANO-GUERRERO J, OLIVAS J A, ROMERO F P, et al. Sentiment analysis: a review and comparative analysis of web services[J]. Information Sciences, 2015, 311: 18-38.
[7] TANG D, QIN B, FENG X, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of the International Conference on Computational Linguistics, 2016: 3298-3307.
[8] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 3rd International Conference on Learning Representations, 2015.
[9] CHEN P, SUN Z, BING L, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017: 452-461.
[10] SONG Y, WANG J, JIANG T, et al. Attentional encoder network for targeted sentiment classification[J]. arXiv:1902.
09314, 2019.
[11] 王家乾, 龚子寒, 薛云, 等. 基于混合多头注意力和胶囊网络的特定目标情感分析[J]. 中文信息学报, 2020, 34(5): 100-110.
WANG J G, GONG Z H, XUE Y, et al. Aspect-based sentiment analysis based on hybrid multi-head attention and capsule networks[J]. Journal of Chinese Information Processing, 2020, 34(5): 100-110.
[12] DEVLIN J, CHANG M, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understand -ing[C]//Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019.
[13] 陈壮, 钱铁云, 李万理, 等. 低资源方面级情感分析研究综述[J]. 计算机学报, 2023, 46(7): 1445-1472.
CHEN Z, QIAN T Y, LI W L, et al. Low-resource aspect-based sentiment analysis: a survey[J]. Chinese Journal of Computers, 2023, 46(7): 1445-1472.
[14] MA D, LI S, ZHANG X, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017: 4068-4074.
[15] LI X, BING L D, LAM W, et al. Transformation networks for target-oriented sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2018.
[16] ZHANG C, LI Q, SONG D, et al. Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019.
[17] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the Twenty-Seventh Annual Conference on Neural Information Processing Systems Conference, 2013: 3111-3119.
[18] PENNINGTON J, SOCHER R, MANNING C. Glove: global vectors for word representation[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2014.
[19] 宋威, 温子健. 基于特征双重蒸馏网络的方面级情感分析[J]. 中文信息学报, 2021, 35(7): 126-133.
SONG W, WEN Z J. Feature dual distillation network for aspect-based sentiment analysis[J]. Journal of Chinese Information Processing, 2021, 35(7): 126-133.
[20] PONTIKI M, GALANIS D, PAPAGEORGIOU H, et al. SemEval2014 task4: aspect based sentiment analysis[C]//Proceedings of the 18th International Workshop on Semantic Evaluation, 2014: 27-35.
[21] 夏鸿斌, 顾艳, 刘渊. 面向特定方面情感分析的图卷积过度注意(ASGCN-AOA)模型[J]. 中文信息学报, 2022, 36(3): 146-153.
XIA H B, GU Y, LIU Y. Graph convolution overattention (ASGCN-AOA) model for specific aspects of sentiment analysis[J]. Journal of Chinese Information Processing, 2022, 36(3): 146-153.
[22] XU L, BING L, LU W, et al. Aspect sentiment classification with aspect-specific opinion spans[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 2020.
[23] SONG Y, WANG J, TAO J, et al. Attentional encoder network for targeted sentiment classification[J]. arXiv:1902.
09314, 2019.
[24] 王娅丽, 张凡, 余增, 等. 基于交互注意力和图卷积网络的方面级情感分析[J]. 计算机科学, 2023, 50(4): 196-203.
WANG Y L, ZHANG F, YU Z, et al. Aspect-level sentiment classification based on interactive attention and graph convolutional network[J]. Computer Science, 2023, 50(4): 196-203.