Chinese Implicit Sentiment Analysis Based on Graph Attention Neural Network

doi:10.3778/j.issn.1002-8331.2007-0100

Abstract

Abstract:

Sentiment analysis is one of important tasks in the field of Natural Language Processing. The task of sentiment analysis consists of implicit sentiment analysis and explicit sentiment analysis. Implicit sentiment suffers greater challenge because sentence does not include explicit emotion words and emotional expression is more euphemistic. Implicit sentiment analysis model based on graph attention convolutional neural network, ISA-GACNN, is proposed, constructing the heterogeneous graph of words and sentences. Graph convolutional neural network is used to propagating semantic information. Attention mechanism is used to compute the contribution to emotional expression of words. In order to solve the problem of multiple attention preserving repeated information, attention orthogonal constraint is used to make different attention store different emotional information. In view of the uneven distribution of emotional information, attention score constraint is proposed to make the model focus on a limited number of important words. The performance of proposed model is verified on implicit sentiment analysis evaluation datasets. The [F] value reaches 91.7 percent, which is higher than the benchmark model in literatures. The attention mechanism is analyzed to verify the effectiveness of orthogonal constraint and score constraint.

Key words: implicit sentiment analysis, attention mechanism, graph neural network, orthogonal constraint

摘要：

情感分析是自然语言处理领域的重要任务之一，情感分析任务包含显式情感分析和隐式情感分析。由于隐式情感不包含显式情感词语，情感表达更加委婉，所以面临更大的挑战。提出基于图注意力神经网络的隐式情感分析模型ISA-GACNN（Implicit Sentiment Analysis Based on Graph Attention Convolutional Neural Network），构建文本和词语的异构图谱，使用图卷积操作传播语义信息，使用注意力机制计算词语对文本情感表达的贡献程度。针对多头注意力保存重复信息问题，使用注意力正交约束使得不同注意力存储不同的情感信息；针对情感信息分布不均的情况，提出注意力分值约束使模型关注部分重要词语。在隐式情感分析评测数据集上验证模型效果，所提出模型的[F]值达到91.7%，远高于文献中的基准模型；对注意力机制进行分析，验证了正交约束和分值约束的有效性。

关键词: 隐式情感分析, 注意力机制, 图神经网络, 正交约束

YANG Shanliang, CHANG Zheng. Chinese Implicit Sentiment Analysis Based on Graph Attention Neural Network[J]. Computer Engineering and Applications, 2021, 57(24): 161-167.

杨善良，常征. 基于图注意力神经网络的中文隐式情感分析[J]. 计算机工程与应用, 2021, 57(24): 161-167.

References

[1] WEI J，LIAO J，YANG Z，et al.BiLSTM with multi-polarity orthogonal attention for implicit sentiment analysis[J].Neurocomputing，2020，383：165-173.
[2] ZUO E，ZHAO H，CHEN B，et al.Context-specific heterogeneous graph convolutional network for implicit sentiment analysis[J].IEEE Access，2020，8：37967-37975.
[3] ZHANG T，HUANG M，ZHAO L.Learning structured representation for text classification via reinforcement learning[C]//Association for the Advancement of Artificial Intelligence，New Orleans，LA，USA，2018：6053-6066.
[4] FANG Z，ZHANG Q，TANG X，et al.An implicit opinion analysis model based on feature?based implicit opinion patterns[J].Artificial Intelligence Review，2020，53（6）.
[5] CHOI Y，WIEBE J，MIHALCEA R.Coarse-grained +/-effect word sense disambiguation for implicit sentiment analysis[J].IEEE Transactions on Affective Computing，2017，8（4）：471-479.
[6] 赵容梅，熊熙，琚生根，等.基于混合神经网络的中文隐式情感分析[J].四川大学学报（自然科学版），2020，57（2）：264-270.
ZHAO R M，XIONG X，JU S G，et al.Implicit sentiment analysis for Chinese texts based on a hybrid neural network[J].Journal of Sichuan University（Natural Science Edition），2020，57（2）：264-270.
[7] ZHOU J，CUI H，ZHANG Z，et al.Graph neural networks：a review of methods and applications[J].arXiv：1812.08434，2018.
[8] ZHANG S，TONG H，XU J，et al.Graph convolutional networks：a comprehensive review[J].Computational Social Networks，2019，6（11）：1-23.
[9] VELICKOVI P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[10] ZHANG Z，SHI X，XIE J，et al.GaAN：gated attention networks for learning on large and spatiotemporal graphs[J].arXiv：1803.07294，2018.
[11] ZHANG C X，SONG D J，HUANG C.Heterogeneous graph neural network[C]//Knowledge Discovery and Data Mining，Anchorage，AK，USA，August 4-8，2019：793-803.
[12] WANG X，JI H，SHI C，et al.Heterogeneous graph attention network[J].arXiv：1903.07293v1，2019.
[13] YAO L，MAO C，LUO Y.Graph convolutional networks for text classification[C]//The Thirty-Third AAAI Conference on Arti?cial Intelligence（AAAI-19），2019：7370-7377.
[14] LIU X，YOU X，ZHANG X，et al.Tensor graph convolutional networks for text classification[J].arXiv：2001.
05313，2020.
[15] CHEN J，HOU H，JI Y，et al.Graph convolutional networks with structural attention model for aspect based sentiment analysis[C]//International Joint Conference on Neural Networks，Budapest，Hungary，14-19 July 2019：1-7.
[16] XIAO L，HU X，CHEN Y，et al.Targeted sentiment classification based on attentional encoding and graph convolutional networks[J].Applied Sciences，2020，10（3）：957.
[17] ZHAO P，HOU L，WU Q.Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification[J].arXiv：1906.04501，2019.
[18] BIJARI K，ZARE H，KEBRIAEI E，et al.Leveraging deep graph-based text representation for sentiment polarity applications[J].arXiv：1902.10247，2019.
[19] VASHISHTH S，BHANDARI M，YADAV P，et al.Incorporating syntactic and semantic information in word embeddings using graph convolutional networks[J].arXiv：1809.
04283，2018.
[20] KNYAZEV B，GRAHAM W T，MOHAMED R A.Understanding attention and generalization in graph neural networks[C]//33rd Conference on Neural Information Processing Systems（NeurIPS 2019），Vancouver，Canada，2019.