Multilayer Gating and Relational Graph Attention Fusion Network for Aspect-Based Sentiment Analysis

doi:10.3778/j.issn.1002-8331.2204-0439

Abstract

Abstract: As the mainstream techniques in current aspect-based sentiment analysis field, serialization neural network and graph neural networks focus on semantic and syntactic relation modeling, respectively. Aiming at the problems that serialization neural network cannot accurately capture the long-distance dependency of complex sentences, and graph neural network lacked local sequence semantics as well as fine-grained final sentiment representation, multilayer gating and relational graph attention fusion network is designed. The pre-trained model ERNIE 2.0 generated contextualized representations. Aspect-based gated recurrent unit captured sequence semantic information about aspect items. Relational graph attention network learned higher-order syntactic features in local sequence semantics. Finally, the feature refinement dual channels consisting of dual refinement gate network strengthen the interaction between specific aspect and context, filter irrelevant contextual information in semantics and syntax, and obtain aspect sentiment-enhanced representation with semantic and syntactic relations. Experiments on Twitter and SemEval2014 datasets show that the proposed fusion network has superior classification performance compared to eight advanced baseline methods.

Key words: aspect-based sentiment analysis, multilayer gating network, relational graph attention network, feature interactive refinement, local feature extraction

摘要： 作为目前方面情感分析领域的主流技术，序列化神经网络和图神经网络分别聚焦于语义和句法关系建模。针对序列化神经网络无法准确捕获复杂句的远距离依赖关系，图神经网络缺乏局部序列语义以及精细的最终情感表达等问题，设计了一种多层门控与关系图注意力混合网络。采用预训练模型ERNIE 2.0生成语境化表示，构建方面门控循环单元捕获有关方面的序列语义信息，使用关系图注意力网络学习局部序列语义中的高阶句法特征。最后由双重蒸馏门控网络构成的特征蒸馏双通道，强化特定方面与上下文之间的交互，过滤语义和句法中的冗余信息，获取兼具语义和句法关系的方面情感增强表示。在Twitter和SemEval2014数据集上进行的实验表明，相较于八种先进基线方法，所提出的混合网络具有更优的分类性能。

关键词: 方面情感分析, 多层门控网络, 关系图注意力网络, 特征交互蒸馏, 局部特征提取

LUO Rongrong, GONG Hongfang, XU Dan. Multilayer Gating and Relational Graph Attention Fusion Network for Aspect-Based Sentiment Analysis[J]. Computer Engineering and Applications, 2023, 59(15): 169-176.

罗容容, 龚红仿, 徐丹. 融合多层门控与关系图注意力的方面情感分析[J]. 计算机工程与应用, 2023, 59(15): 169-176.

References

[1] LIU B.Sentiment analysis and opinion mining[J].Synthesis Lectures on Human Language Technologies，2012，5（1）：1-167.
[2] HUANG B X，OU Y L，CARLEY K M.Aspect level sentiment classification with attention-over-attention neural networks[C]//Proceedings of International Conference on Social Computing，Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation，2018：197-206.
[3] HOU X C，HUANG J，WANG G T，et al.Selective attention based graph convolutional networks for aspect-level sentiment classification[C]//Proceedings of the 15th Workshop on Graph-Based Methods for Natural Language Processing（TextGraphs-15），2021：83-93.
[4] XIAO L W，XUE Y，WANG H，et al.Exploring fine-grained syntactic information for aspect-based sentiment classification with dual graph neural networks[J].Neuro-computing，2022，471：48-59.
[5] BAI X F，LIU P B，ZHANG Y.Investigating typed syntactic dependencies for targeted sentiment classification using graph attention neural network[J].IEEE/ACM Transactions on Audio，Speech，and Language Processing，2021，29：503-514.
[6] YANG H，ZENG B Q，YANG J H，et al.A multi-task learning model for Chinese-oriented aspect polarity classification and aspect term extraction[J].Neurocomputing，2021，419：344-356.
[7] XU M Y，ZENG B Q，YANG H，et al.Combining dynamic local context focus and dependency cluster attention for aspect-level sentiment classification[J].Neurocomputing，2022，478：49-69.
[8] WANG Y Q，HUANG M L，ZHU X Y，et al.Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，2016：606-615.
[9] ZENG J F，MA X，ZHOU K.Enhancing attention-based LSTM with position context for aspect-level sentiment classification[J].IEEE Access，2019，7：20462-20471.
[10] MA D H，LI S J，ZHANG X D，et al.Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence，2017：4068-4074.
[11] ZHANG M S，ZHANG Y，VO D T.Gated neural net-works for targeted sentiment analysis[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence，2016：3087-3093.
[12] 张瑾，段利国，李爱萍，等.基于注意力与门控机制相结合的细粒度情绪分析[J].计算机科学，2021，48（8）：226-233.
ZHANG J，DUAN L G，LI A P，et al.Fine-grained sentiment analysis based on combination of attention and gated mechanism[J].Computer Science，2021，48（8）：226-233.
[13] LIU N，SHEN B.Aspect-based sentiment analysis with gated alternate neural network[J].Knowledge-Based Systems，2020，188：105010.
[14] RAN X Y，PAN Y Y，SUN W，et al.Learn to select via hierarchical gate mechanism for aspect-based sentiment analysis[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence（IJCAI-19），2019：5160-5167.
[15] CAI H J，TU Y F，ZHOU X S，et al.Aspect-category based sentiment analysis with hierarchical graph convolutional network[C]//Proceedings of the 28th International Conference on Computational Linguistics，2020：833-843.
[16] 肖泽管，陈清亮.融合多种类型语法信息的属性级情感分析模型[J].计算机科学与探索，2022，16（2）：395-402.
XIAO Z G，CHEN Q L.Aspect-based sentiment analysis model with multiple grammatical information[J].Journal of Frontiers of Computer Science and Technology，2022，16（2）：395-402.
[17] CHEN B L，LU G J，XUE Y，et al.A novel bi-branch graph convolutional neural network for aspect level sentiment classification[C]//Proceedings of 2021 International Joint Conference on Neural Networks（IJCNN），2021：1-8.
[18] VELICKOVIC P，CUCURULL G，ROMERO A，et al.Graph attention networks[C]//Proceedings of International Conference on Learning Representations，2018.
[19] HUANG B X，CARLEY K.Syntax-aware aspect level sentiment classification with graph attention networks[C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：5469-5477.
[20] HOU X C，QI P，WANG G T，et al.Graph ensemble learning over multiple dependency trees for aspect-level sentiment classification[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2021：2884-2894.
[21] SUN Y，WANG S H，LI Y K，et al.ERNIE 2.0：a continual pre-training framework for language understanding[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：8968-8975.
[22] SONG W，WEN Z J，XIAO Z Y，et al.Semantics perception and refinement network for aspect-based sentiment analysis[J].Knowledge-Based Systems，2021，214：106755.
[23] CHO K，MERRIENBOER B V，GULCEHRE C，et al.Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing（EMNLP），2014：1724-1734.
[24] TANG D Y，QIN B，FENG X C，et al.Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of COLING 2016，the 26th International Conference on Computational Linguistics：Technical Papers，2016：3298-3307.
[25] SONG Y W，WANG J H，JIANG T，et al.Targeted sentiment classification with attentional encoder network[C]//Proceedings of the International Conference on Artificial Neural Networks，2019：93-103.
[26] ZHANG C，LI Q C，SONG D W.Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing，2019：4567-4577.