Aspect-Based Sentiment Analysis with Cross-Heads Attention

doi:10.3778/j.issn.1002-8331.2201-0454

Abstract

Abstract: Aspect level affective analysis aims to identify the positive, negative and neutral emotions of aspect words in sentences. The key is to learn the relationship between aspect words and words in sentences. When learning the relationship between words, the existing convolution gated network uses the time convolution method, and its local time window can not describe the relationship between any words. At the same time, the attention of the existing temporal attention model is independent of each other when analyzing the relationship between words. In order to analyze the complex relationship between aspect words and other words in sentences, an emotion analysis model based on cross attention and convolution gated network is proposed in this paper. Firstly, for a given word vector feature, this paper designs a cross attention module. The module adds crossed linear mapping to the matching scores of query vector and keyword vector in multiple attention, so as to integrate the matching scores in multiple attention, which is used to describe the context word relationship of more complex aspect words. Secondly, this paper uses the gated convolution network to encode the local word relationship, and designs the word position coding module to provide the position coding characteristics of words, so as to analyze the effect of position coding on the analysis of word relationship. Finally, for the above encoded word features, this paper uses time pooling to obtain sentence description, and uses full connection classifier to predict emotion classification markers. The experimental analysis on Rest14 and Laptop14 data sets shows that this method can effectively estimate the score relationship between aspect words and other words.

Key words: aspected-based sentiment analysis, gating mechanism, cross-heads attention, position information

摘要： 方面级情感分析旨在识别句子中方面词的积极、消极和中性情绪。其关键在于方面词和句子中单词之间关系的学习。在学习单词之间关系时，现有卷积门控网络使用时间卷积方法，其局部时间窗口无法描述任意单词之间的关系。同时，现有时间注意力模型在分析单词之间的关系时，其注意力是相互独立的。为了分析句子中方面词与其他单词的复杂关联，提出一种基于交叉注意力和卷积门控网络的情感分析模型。对于给定的词向量特征，设计了一种交叉注意力模块。该模块对多头注意力中查询向量与关键字向量的匹配得分，添加交叉的线性映射，以融合多个注意力中的匹配得分，用于描述更复杂的方面词的上下文单词关系。使用卷积门控网络对局部单词关系进行编码，并设计了单词的位置编码模块，用于提供单词的位置编码特征，以分析位置编码对单词关系分析的作用。对上述编码的单词特征，使用时间池化获得句子描述，并使用全连接分类器进行情感分类标记预测。在Rest14和Laptop14数据集上的实验分析表明，提出的方法能有效估计方面级单词与其他单词之间得分关系。

关键词: 方面级情感分析, 门控机制, 交叉注意力, 位置信息

ZHOU Runmin, HU Xuyao, WU Kewei, YU Lei, XIE Zhao, JIANG Long. Aspect-Based Sentiment Analysis with Cross-Heads Attention[J]. Computer Engineering and Applications, 2023, 59(9): 190-197.

周润民, 胡旭耀, 吴克伟, 于磊, 谢昭, 江龙. 基于交叉注意力的方面级情感分析[J]. 计算机工程与应用, 2023, 59(9): 190-197.

References

[1] LIU N，SHEN B.Aspect-based sentiment analysis with gated alternate neural network[J].Knowledge-Based Systems，2020，188：105010.
[2] 石磊，张鑫倩，陶永才，等.结合自注意力机制和Tree-LSTM的情感分析模型[J].小型微型计算机系统，2019，40（7）：1486-1490.
SHI L，ZHANG X Q，TAO Y C，et al.Sentiment analysis model with the combination of self-attention and tree-LSTM[J].Journal of Chinese Computer Systems，2019，40（7）：1486-1490.
[3] SONG Y，HU Q V，HE L.P-CNN：enhancing text matching with positional convolutional neural network[J].Knowledge-Based Systems，2019，169：67-79.
[4] WEI X，TAO L.Aspect based sentiment analysis with gated convolutional networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，Melbourne，July 15-20，2018.Stroudsburg：ACL，2018：2514-2523.
[5] ZHANG C，LI Q，SONG D.Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，Hong Kong，Nov 3-7，2019.Stroudsburg：ACL，2019：4567-4577.
[6] YANG Jian，YANG Juan.Aspect based sentiment analysis with self-attention and gated convolutional networks[C]//2020 IEEE 11th International Conference on Software Engineering and Service Science，Beijing，Oct 16-18，2020.New York：IEEE，2020：146-149.
[7] WAN Y，NIE H，LAN T，et al.Fine-grained sentiment analysis of online reviews[C]//12th International Conference on Fuzzy Systems and Knowledge Discovery，Zhangjiajie，Aug 15-17，2015.New York：IEEE，2016：1406-1411.
[8] MEDHAT W，HASSAN A，KORASHY H.Sentiment analysis algorithms and applications：a survey[J].Ain Shams Engineering Journal，2014，5（4）：1093-1113.
[9] WEI X，ZHOU W，LI T，et al.MTNA：a neural multi-task model for aspect category classification and aspect term extraction on restaurant reviews[C]//Proceedings of the Eighth International Joint Conference on Natural Language Processing，Taipei，China，Nov 27-Dec 1，2017：151-156.
[10] ZHANG C，LI Q，SONG D.Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network[J].arXiv：1909.10171，2019.
[11] WANG K，SHEN W，YANG Y，et al.Relational graph attention network for aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistic，July 5-10，2020.Stroudsburg：ACL，2020：3229-3238.
[12] XU Q，ZHU L，DAI T，et al.Aspect-based sentiment classification with multi-attention network[J].Neurocomputing，2020，388：135-143.
[13] HU M，ZHAO S，LI Z，et al.CAN：constrained attention networks for multi-aspect sentiment analysis[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，Hong Kong，Nov 3-7，2018.Stroudsburg：ACL，2019：4600-4609.
[14] SONG Y，WANG J，JIANG T，et al.Targeted sentiment classification with attentional encoder network[C]//Artificial Neural Networks and Machine Learning，Germany，Sep 17-19，2019.Germany：Springer，2019：93-103.
[15] ZHAO F，WU Z，DAI X.Attention transfer network for aspect-level sentiment classification[C]//Proceedings of the 28th International Conference on Computational Linguistics，Barcelona，Dec 8-13，2020.France：ICCL，2020：811-821.
[16] LIU M Z，ZHOU F Y，CHEN K，et al.Co-attention networks based on aspect and context for aspect-level sentiment analysis[J].Knowledge-Based Systems，2021，217（2）：106810.
[17] SONG Y，WANG J，TAO J，et al.Attentional encoder network for targeted sentiment classification[EB/OL].[2019-03-21].http：//arxiv.org/abs/1902.09314.
[18] LIU Q，MUKAIDANI H.Effective-target representation via lstm with attention for aspect-level sentiment analysis[C]//2020 International Conference on Artificial Intelligence in Information and Communication，Fukuoka，Feb 19-21，2020.New York：IEEE，2020：336-340.
[19] GUO Q，QIU X，LIU P，et al.Multi-scale self-attention for text classification[C]//The Thirty-Fourth Conference on Artificial Intelligence，New York，Feb 7-12，2020.New York：AAAI，2020：7847-7854.
[20] LI J，WANG X，TU Z，et al.On the diversity of multi-head attention[J].Neurocomputing，2021，454：14-24.
[21] CHEN Y，ZHUANG T，GUO K.Memory network with hierarchical multi-head attention for aspect-based sentiment analysis[J].Applied Intelligence，2021，51（1）：1-18.
[22] SHAO D，AN Q，HUANG K，et al.Aspect-level sentiment analysis for based on joint aspect and position hierarchy attention mechanism network[J].Journal of Intelligence and Fuzzy Systems，2022，42（3）：2207-2218.
[23] LIN T，SUN A，WANG Y.Aspect-based sentiment analysis through edu-level attentions[C]//The Pacific-Asia Conference on Knowledge Discovery and Data Mining，2022：156-168.
[24] DAUPHIN Y N，FAN A，AULI M，et al.Language modeling with gated convolutional networks[C]//Proceedings of the 34th International Conference on Machine Learning，Sydney，Aug 6-11，2017.Australia：PMLR，2017：933-941.
[25] LIU Y，JI L X，HUANG R，et al.A multi-sense word embedding method based on gated convolution and hierarchical attention mechanism[J].Journal of Chinese Information Processing，2018.
[26] LIANG Y，MENG F，ZHANG J，et al.A novel aspect-guided deep transition model for aspect based sentiment analysis[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing，Hong Kong，Nov 3-7，2019.Stroudsburg：ACL，2019：5568-5579.
[27] PENNINGTON J，SOCHER R，MANNING C.Glove：global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，Doha，Oct 25-29，2014.Stroudsburg：ACL，2014：1532-1543.
[28] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[EB/OL].[2017-12-06].https：//arxiv.org/pdf/1706.03762.pdf.
[29] PONTIKI M，GALANIS D，PAVLOPOULOS J，et al.SemEval-2014 Task 4：aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation，SemEval@COLING 2014，Dublin，Aug 23-24，2014.Stroudsburg：ACL，2014：27-35.