Multiple Classification Method for Microblog Negative Emotions Integrating MAML and BiLSTM

doi:10.3778/j.issn.1002-8331.2009-0337

Abstract

Abstract: With the continuous development of social networks, microblog has become an important platform for people to share their opinions and feelings in daily life. The analysis of users’ emotional tendencies can be effectively applied to public opinion control, public opinion survey, commodity recommendation and other work. When faced with new tasks, the traditional deep learning algorithm often needs a large amount of data retraining to get a good accuracy. In view of this situation, this study proposes a microblog negative emotion multi-classification method based on MAML and BiLSTM. Firstly, the word vectioning of microblog text is quantified, and then the model combining MAML and BiLSTM is constructed. BiLSTM realizes the classification of negative emotions on microblog and updates the parameters through random gradient descent. The meta-learner in MAML updates its parameters by calculating the sum of the losses from multiple trainings and performing the second gradient descent. Finally, the learner obtained through the update can quickly iterate when faced with the new negative emotion classification task of microblog. The experimental results show that, compared with the current popular model, the accuracy, recall rate and F1-score of microblog negative emotion data set are improved by 1.68 percentage points, 2.86 percentage points and 2.27 percentage points, respectively.

Key words: bi-directional long short-term memory（BiLSTM）, model-agnostic meta-learning（MAML）, microblog, sentiment analysis

摘要： 随着社交网络的不断发展，微博成为人们日常生活中分享观点和感情的重要平台，分析用户的情感倾向可以有效地应用于舆情控制、民意调查、商品推荐等工作。传统的深度学习算法在面对新的工作任务时，往往需要大量数据重新训练才能得到较好准确率。针对这一情况，提出了一种基于MAML（model-agnostic meta-learning）与BiLSTM（双向长短时记忆网络）的微博负面情感多分类方法。对微博文本进行词向量化表示，构建MAML与BiLSTM结合的模型，其中BiLSTM实现对微博负面情感的分类，通过随机梯度下降更新参数；MAML中的元学习器则通过计算多次训练的损失总和，进行第二次梯度下降，更新元学习器参数。通过更新后得到的元学习器可以在面对新的微博负面情感分类任务时快速迭代。实验结果表明：相较于目前流行的模型，在微博负面情感数据集上，准确率、召回率和F1值分别提高了1.68个百分点、2.86个百分点和2.27个百分点。

关键词: 双向长短时记忆网络（BiLSTM）, MAML, 微博, 情感分析

XU Chao, YE Ning, XU Kang, WANG Ruchuan. Multiple Classification Method for Microblog Negative Emotions Integrating MAML and BiLSTM[J]. Computer Engineering and Applications, 2022, 58(5): 179-185.

徐超, 叶宁, 徐康, 王汝传. 融合MAML与BiLSTM的微博负面情感多分类方法[J]. 计算机工程与应用, 2022, 58(5): 179-185.

References

[1] NASUKAWA T，YI J.Sentiment analysis：capturing favorability using natural language processing[C]//Proceedings of the 2nd International Conference on Knowledge Capture.New York，NY，USA：ACM，2003.
[2] SOCHER R，PERELYGIN A，WU J，et al.Recursive deep models for semantic compositionality over a sentiment treebank[C]//Conference on Empirical Methods in Natural Language Processing，Washington，2013：1631-1642.
[3] WANG J，YU L C，LAI K R，et al.Dimensional sentiment analysis using a regional CNN-LSTM model[C]//Proceedings of the Annual Meeting of the Association for Computational Linguistics，Berlin，Germany，2016：225-230.
[4] JIANG N，TIAN F，LI J，et al.MAN：mutual attention neural networks model for aspect-level sentiment classification in SIoT[J].IEEE Internet of Things Journal，2020，7（4）：2901-2913.
[5] ZHAO P，HOU L，WU O.Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification[J].Knowledge-Based Systems，2020，193：105443.
[6] SCHMIDHUBER J.Evolutionary principles in self-referential learning. on learning now to learn：the meta-meta-meta...-hook[D].Germany：Technische Universitat Munchen，1987.
[7] BENGIO S，BENGIO Y，CLOUTIER J，et al.On the optimization of a synaptic learning rule[C]//Preprints Conf Optimality in Artificial and Biological Neural Networks，1992.
[8] GILES C L，HANSON S J，COWAN J D.Explanation-based neural network learning for robot control[C]//Advances in Neural Information Processing Systems，1993：287-294.
[9] VILALTA R，DRISSI Y.A perspective view and survey of meta-learning[J].Artificial Intelligence Review，2002，18（2）：77-95.
[10] THRUN S.Lifelong learning algorithms[M]//Learning to learn.Boston，MA：Springer，1998：181-209.
[11] GRAVES A，WAYNE G，DANIHELKA I.Neural turing machines[J].arXiv：1410.5401，2014.
[12] SANTORO A，BARTUNOV S，BOTVINICK M，et al.Meta-learning with memory-augmented neural networks[C]//International Conference on Machine Learning，2016：1842-1850.
[13] RAVI S，LAROCHELLE H.Optimization as a model for few-shot learning[C]//Proceedings of ICLR，2016.
[14] FINN C，ABBEEL P，LEVINE S.Model-agnostic meta-learning for fast adaptation of deep networks[C]//International Conference on Machine Learning，2017：1126-1135.
[15] MUNKHDALAI T，YU H.Meta networks[C]//International Conference on Machine Learning，2017：2554-2563.
[16] XU J，TON J F，KIM H，et al.Metafun：meta-learning with iterative functional updates[C]//International Conference on Machine Learning，2020：10617-10627.
[17] REN M，TRIANTAFILLOU E，RAVI S，et al.Meta-learning for semi-supervised few-shot classification[J].arXiv：1803.00676，2018.
[18] COLLINS L，MOKHTARI A，SHAKKOTTAI S.Task-robust model-agnostic meta-learning[C]//Advances in Neural Information Processing Systems，2020.
[19] 栗雨晴，礼欣，韩煦，等.基于双语词典的微博多类情感分析方法[J].电子学报，2016，44（9）：2068-2073.
LI Y Q，LI X，HAN X，et al.A bilingual lexicon-based multi-class semantic orientation analysis for microblogs[J].Acta Electronica Sinica，2016，44（9）：2068-2073.
[20] 刘志明，刘鲁.基于机器学习的中文微博情感分类实证研究[J].计算机工程与应用，2012，48（1）：1-4.
LIU Z M，LIU L.Empirical study of sentiment classification for Chinese microblog based on machine learning[J].Computer Engineering and Applications，2012，48（1）：1-4.
[21] SANTOS C N，GATTI M A.Deep convolutional neural network for sentiment analysis of short text[C]//Proc of COLING，2014：23-29．
[22] YAO Yushi，HUANG Zheng.Bi-directional LSTM recurrent neural network for Chinese word segmentation[M]//Neural information processing.Berlin，Germany：Springer International Publishing，2016.
[23] ABDI A，SHAMSUDDIN S M，HASAN S，et al.Deep learning-based sentiment classification of evaluative text based on multi-feature fusion[J].Information Processing & Management，2019，56（4）：1245-1259.
[24] CHEN Z，QIAN T.Transfer capsule network for aspect level sentiment classification[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：547-556.
[25] BERMINGHAM A，SMEATON A F.Classifying sentiment in microblogs：is brevity an advantage?[C]//Proceedings of the 19th ACM International Conference on Information and Knowledge Management，2010：1833-1836.
[26] KIM Y.Convolutional neural networks for sentence classification[J].arXiv：1408.5882，2014.
[27] ZAREMBA W，SUTSKEVER I，VINYALS O.Recurent neural network regularization[J].arXiv：1409.2329，2014.
[28] VINYALS O，BLUNDELL C，LILLICRAP T，et al.Matching networks for one shot learning[J].arXiv：1606.04080，2016.