Aspect-Level Sentiment Analysis Research Integrating Dependent Syntactic Prior Knowledge

doi:10.3778/j.issn.1002-8331.2203-0215

Abstract

Abstract: In view of the insufficient utilization of dependent syntactic information in aspect-level sentiment analysis, as well as the separate learning of contextual information and dependent syntactic information, a research method of aspect-level sentiment analysis integrating prior knowledge and pre-training model is proposed. Firstly, this method inputs the dependent syntax information as prior knowledge and the sentence into the pre-training model. Through the pre-training model, the long-distance information between aspect words and emotional words of dependent syntactic modeling is learned while learning the context information of the sentence. Secondly, the dependency distance representation is further used to match the dynamic weight of the pre-training word vector, so as to make full use of the syntactic information. Thirdly, the part of speech information and the dependency label information are learned through self-attention mechanism, and using two auxiliary information to enhance sentence representation. Lastly, three kinds of information are spliced as the input of sentiment classifier. Compared with other aspect-level sentiment analysis methods, the accuracy and F1 value are increased on the four benchmark data sets, which verifies the effectiveness of prior knowledge fusion and dynamic weight matching proposed by this method, and has a great application value in aspect-level sentiment analysis tasks.

Key words: dependency syntax, prior knowledge, pre-training model, dynamic weight matching, self-attention mechanism

摘要： 针对方面级情感分析中依存句法信息利用不足，以及上下文信息与依存句法信息学习分离的问题，提出融合先验知识与预训练模型的方面级情感分析研究方法。该方法将依存句法信息作为先验知识与句子一起输入到预训练模型，通过预训练模型在对句子进行上下文信息学习的同时，学习依存句法建模的方面词与情感词之间的远距离信息。进一步利用依存距离表示将预训练词向量进行动态权重匹配从而充分利用句法信息。通过自注意力机制学习词性信息以及依存关系标签信息，通过两种辅助信息增强句子表示。拼接三种信息作为情感分类器的输入。在四种基准数据集上与其他方面级情感分析方法相比，准确率和F1值都有所提高，验证了该方法提出的先验知识融合与动态权重匹配的有效性，在方面级情感分析任务上有较大的应用价值。

关键词: 依存句法, 先验知识, 预训练模型, 动态权重匹配, 自注意力机制

FANG Yiqiu, PENG Yang, GE Junwei. Aspect-Level Sentiment Analysis Research Integrating Dependent Syntactic Prior Knowledge[J]. Computer Engineering and Applications, 2023, 59(12): 122-131.

方义秋, 彭杨, 葛君伟. 融合依存句法先验知识的方面级情感分析研究[J]. 计算机工程与应用, 2023, 59(12): 122-131.

References

[1] JIANG L，YU M，ZHOU M，et al.Target-dependent twitter sentiment classification[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics，Portland，2011：151-160.
[2] KIRITCHENKO S，ZHU X，CHERRY C，et al.Nrc-canada-2014：detecting aspects and sentiment in customer reviews[C]//Proceedings of the 8th International Workshop on Semantic Evaluation，Montreal，2014：437-442.
[3] TANG D，QIN B，FENG X，et al.Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of the 26th International Conference on Computational Linguistics，Osaka，2016：3298-3307.
[4] XUE W，LI T.Aspect based sentiment analysis with gated convolutional networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，Melbourne，2018：2514-2523.
[5] WANG Y，HUANG M，ZHU X，et al.Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，Texas，2016：606-615.
[6] MA D，LI S，ZHANG X，et al.Interactive attention networks for aspect-level sentiment classification[C]//Twenty-Sixth International Joint Conference on Artificial Intelligence，Melbourne，2017.
[7] FAN F，FENG Y，ZHAO D.Multi-grained attention network for aspect-level sentiment classification[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，Brussels，2018：3433-3442.
[8] WANG B，LU W.Learning latent opinions for aspect-level sentiment classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence，New Orleans，2018：5537-5544.
[9] LEI Z，YANG Y，YANG M，et al.A human-like semantic cognition network for aspect-level sentiment classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence，Hawaii，2019：6650-6657.
[10] LI X，BING L，LAM W，et al.Transformation networks for target-oriented sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，Melbourne，2018：946-956.
[11] TANG J，LU Z，SU J，et al.Progressive self-supervised attention learning for aspect-level sentiment analysis[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，Florence，2019：557-566.
[12] TANG D，QIN B，LIU T.Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，Austin，2016：214-224.
[13] CHEN P，SUN Z，BING L，et al.Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing，Copenhagen，2017：452-461.
[14] WANG S，MAZUMDER S，LIU B，et al.Target-sensitive memory networks for aspect sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，Melbourne，2018：957-967.
[15] HE R，LEE W，NG H，et al.Exploiting document knowledge for aspect-level sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，Melbourne，2018：579-585.
[16] LI Z，WEI Y，ZHANG Y，et al.Exploiting coarse-to-fine task transfer for aspect-level sentiment classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence，Hawaii，2019：4253-4260.
[17] CHEN Z，QIAN T.Transfer capsule network for aspect level sentiment classification[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，Florence，2019：547-556.
[18] 袁勋，刘蓉，刘明.融合多层注意力的方面级情感分析模型[J].计算机工程与应用，2021，57（22）：147-152.
YUAN X，LIU R，LIU M.Aspect-level sentiment analysis model incorporating multi-layer attention[J].Computer Engineering and Applications，2021，57（22）：147-152.
[19] 胡新荣，王哲，刘军平，等.基于多层注意力机制的服装电商评论情感分析[J].计算机技术与发展，2022，32（1）：67-72.
HU X R，WANG Z，LIU J P.Sentiment analysis of clothing e-commerce reviews based on BiGRU-SD-Attention[J].Computer Technology and Development，2022，32（1）：67-72.
[20] KüBLER S，MCDONALD R，NIVRE J.Dependency parsing[J].Synthesis Lectures on Human Language Technologies，2009，1（1）：1-127.
[21] CHEN D，MANNING C.A fast and accurate dependency parser using neural networks[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，Doha，2014：740-750.
[22] DONG L，WEI F，TAN C，et al.Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics，Baltimore，2014：49-54.
[23] HE R，LEE W，NG H，et al.Effective attention modeling for aspect-level sentiment classification[C]//Proceedings of the 27th International Conference on Computational Linguistics，Santa Fe，2018：1121-1131.
[24] ZHANG C，LI Q，SONG D.Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval，Paris，2019：1145-1148.
[25] ZHENG Y，ZHANG R，MENSAH S，et al.Replicate，walk，and stop on syntax：an effective neural network model for aspect-level sentiment classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：9685-9692.
[26] XIAO Z，WU J，CHEN Q，et al.Bert4gcn：using bert intermediate layers to augment GCN for aspect-based sentiment classification[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing，Bangkok，2021：9193-9200.
[27] 陈德刚，艾孜尔古丽.基于注意力的多通道特征融合情感分析[J].计算机工程与设计，2022，43（2）：546-552.
CHEN D G，AZRA GUL.Triple-channel feature mixed sentiment analysis based on attention[J].Computer Engineering and Design，2022，43（2）：546-552.
[28] JACOB D，CHANG M，KENTON L，et al.Bert：pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，Minneapolis，Minnesota，2019：4171-4186.
[29] LOSHCHILOV I，HUTTER F.Fixing weight decay regularization in adam[J].arXiv：1711.05101，2017.
[30] SRIVASTAVA N，HINTON G，KRIZHEVSKY A，et al.Dropout：a simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research，2014，15（1）：1929-1958.
[31] SZEGEDY C，VANHOUCKE V，IOFFE S，et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：2818-2826.