Implicit Sentiment Analysis Based on Context Aware Tree Recurrent Neutral Network

doi:10.3778/j.issn.1002-8331.2010-0029

Abstract

Abstract: Aiming at the problems of inaccurate feature extraction and gradient explodes or gradient disappears of text information in Chinese implicit sentiment analysis by existing serialization models, a parallel hybrid model of bidirectional long short time neural network and context-aware tree recurrent neural network（CA-TRNN） is proposed. Firstly, it is used to extract the context information from the text by BiLSTM, and is extracted the semantic feature information of the target sentence in the text by the tree recurrent neural network（TRNN）. Finally, two representations are fused by the attention mechanism of the specific target sentence, and then the sentiment classification results of the text are obtained by using softmax. The experimental results in the Chinese implicit sentiment analysis task of SMP2019 micro blog show that the model CA-TRNN can effectively improve the accuracy of classification results with low time cost and better application ability.

Key words: context-aware, attention mechanism, tree recurrent neutral network（TRNN）, implicit sentiment analysis

摘要： 针对现有的序列化模型对中文隐式情感分析中特征信息提取不准确以及对篇章级的文本信息提取存在的梯度爆炸或者梯度消失的问题，提出了双向长短时神经网络和上下文感知的树形递归神经网络（context-aware tree recurrent neutral network，CA-TRNN）的并行混合模型。该模型分别利用双向循环长短时记忆神经网络（BiLSTM）提取文本中的上下文信息，树形递归神经网络（TRNN）提取文本中目标句的语义特征信息，最后，使用特定目标句的注意力机制将两个表示信息进行融合表示后，经过softmax得出文本的情感分类结果。采用SMP2019微博中文隐式情感分析任务中的数据进行验证，实验结果表明，所使用的模型（CA-TRNN）可以有效提高分类结果的准确度，时间代价小，具有更好的应用能力。

关键词: 上下文感知, 注意力机制, 树形递归神经网络（TRNN）, 隐式情感分析

CHEN Qiuchang, ZHAO Hui, ZUO Enguang, ZHAO Yuxia, WEI Wenyu. Implicit Sentiment Analysis Based on Context Aware Tree Recurrent Neutral Network[J]. Computer Engineering and Applications, 2022, 58(7): 167-175.

陈秋嫦, 赵晖, 左恩光, 赵玉霞, 魏文钰. 上下文感知的树递归神经网络下隐式情感分析[J]. 计算机工程与应用, 2022, 58(7): 167-175.

References

[1] LIU B.Sentiment analysis：Mining opinions，sentiments，and emotions[M].Cambridge：Cambridge University Press，2015.
[2] LIU B.Sentiment analysis and opinion mining[J].Synthesis Lectures on Human Language Technologies，2012，5（1）：1.
[3] JIANG M X，MAN L，WU Y B.ECNU at SemEval-2017 task 5：An ensemble of regression algorithms with effective features for fine-grained sentiment analysis in financial domain[C]//Proceedings of the 11th International Workshop on Semantic Evaluation，2017：888-893.
[4] DENG S Y，SINHA A P，ZHAO H M.Adapting sentiment lexicons to domain-specific social media texts[J].Decision Support Systems，2017，94：65-76.
[5] FIRTH J R.A synopsis of linguistic theory，2930-1955 in studies in linguistic analysis[J].Transactions of the Philological Society，1957（1）.
[6] LANDAUER T K，PETER W F，LAHAM D.An introduction to latent semantic analysis[J].Discourse Processes，1998，25（2/3）：259-284.
[7] 廖健.基于表示学习的事实型隐式情感分析研究[D].太原：山西大学，2018.
LIAO J.Research on fact-implied implicit sentiment analysis based on representation learning[D].Taiyuan：Shanxi University，2018.
[8] LIAO J，WANG S G，LI D Y.Identification of fact-implied implicit sentiment based on multi-level semantic fused representation[J].Knowledge-Based Systems，2019，165：197-207.
[9] EKATERINA S，SIMONE T，ANNA K.Statistical metaphor processing[J].Computational Lingus，2013，39（2）：301-353.
[10] CARVALHO P，SARMENTO L，SILVA M J，et al.Clues for detecting irony in user?generated contents：oh…!! It’s so easy[C]//Proceedings of the 1st International CIKM Workshop on Topic-Sentiment Analysis for Mass Opinion，2009：53-56.
[11] 殷树林.现代汉语反问句研究[D].福州：福建师范大学，2006.
YIN S L.A study of rhetorical questions in modern Chinese[D].Fuzhou：Fujian Normal University，2006.
[12] ZHANG Y，ZHANG Y.Tree communication models for sentiment analysis[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019.
[13] KIM Y.[HD1] convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，2014.
[14] KALCHBRENNER N，GREFENSTETTE E，BLUNSOM P.A convolutional neural network for modelling sentences[J].arXiv：1404.2188，2014.
[15] MOU L L，PENG H，LI G，et al.Discriminative neural sentence modeling by tree-based convolution[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing，2015.
[16] TAI K S，SOCHER R，MANNING C D.Improved semantic representations from tree-structured long short-term memory networks[C]//Proceedings of the International Joint Conference on Natural Language Processing，2015.
[17] TENG Z Y，ZHANG Y.Head-lexicalized bidirectional tree LSTMs[J].Transactions of the Association for Computational Lingus，2017，5（2）：163-177.
[18] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[C]//Proceedings of International Conference on Learning Representations，2017.
[19] MARCHEGGIANI D，TITOV I.Encoding sentences with graph convolutional networks for semantic role labeling[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing，2017.
[20] BECK D，HAFFARI G，COHN T.Graph-to-sequence learning using gated graph neural networks[C]//Proceedings of the Association for Computational Linguistics Meeting，2018.
[21] ZHANG Y，LIU Q，SONG L F.Sentence-state LSTM for text representation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics，2018.
[22] SONG L F，ZHANG Y，WANG Z G，et al.A graph-to-sequence model for AMR-to-text generation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguisticss，2018.
[23] PAULUS R，SOCHER R，MANNING C D.Global belief recursive neural networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems，2014：2888-2896.
[24] MIWA M，BANSAL M.End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the Association for Computational Linguistics，2016.
[25] DING X W，LIU B，YU P S.A holistic lexicon-based approach to opinion mining[C]//Proceedings of the 2008 International Conference on Web Search and Data Mining，2008.
[26] JIANG L，YU M，ZHOU M，et al.Target-dependent twitter sentiment classification[C]//Meeting of the Association for Computational Linguistics：Human Language Technologies，2012.
[27] BALAHUR A，HERMIDA J M，MONTOYO A.Detecting implicit expressions of emotion in text：A comparative analysis[J].Decision Support Systems，2012，53（4）：742-753.
[28] WALLACE B C，CHOE D K，CHARNIAK E，et al.Sparse，contextually informed models for irony detection：Exploiting user communities，entities and sentiment[C]//Proceedings of the International Joint Conference on Natural Language Processing，2015：1035-1044.
[29] WALLACE B C，CHOE D K，KERTZ L，et al.Humans require context to infer ironic intent[C]//Proceedings of Meeting of the Association for Computational Linguistics，2014：512-516.
[30] BAMMAN D，SMITH N A.Contextualized sarcasm detection on twitter[C]//Proceedings of National Conference on Artificial Intelligence，2015：574-577.
[31] DEVLIN J，CHANG M W，LEE K，et al.BERT：Pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[32] TANG D Y，QIN B，FENG X C，et al.Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of International Conference on Computational Linguistics，2016.
[33] SONG Y W，WANG J H，JIANG T，et al.Targeted sentiment classification with attentional encoder network[C]//Proceedings of International Conference on Artificial Neural Networks，2019.
[34] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[35] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Proceedings of the NIPS，2017：5998-6008.
[36] LIANG B，DU J C，XU R F，et al.Context-aware embedding for targeted aspect-based sentiment analysis[C]//Proceedings of the Association for Computational Linguistics，2019.