基于ALBERT-AFSFN的中文短文本情感分析

doi:10.3778/j.issn.1002-8331.2011-0056

摘要/Abstract

摘要： 针对传统的卷积神经网络未能充分利用不同通道间的文本特征语义信息和关联信息，以及传统的词向量表示方法采用静态方式对文本信息进行提取，忽略了文本的位置信息，从而导致文本情感分类不准确的问题，提出了一种结合ALBERT（a lite BERT）和注意力特征分割融合网络（attention feature split fusion network，AFSFN）的中文短文本情感分类模型ALBERT-AFSFN。该模型利用ALBERT对文本进行词向量表示，提升词向量的表征能力；通过注意力特征分割融合网络将特征分割为两组，对两组不同通道的特征进行提取和融合，最大程度保留不同通道之间的语义关联信息；借助Softmax函数对中文短文本情感进行分类，得到文本的情感倾向。在三个公开数据集Chnsenticorp、waimai-10k和weibo-100k上的准确率分别达到了93.33%、88.98%和97.81%，F1值也分别达到了93.23%、88.47%和97.78%，结果表明提出的方法在中文短文本情感分析中能够达到更好的分类效果。

关键词: ALBERT, 分割注意力, 特征融合, 情感分析

Abstract: The traditional convolution neural network can not make full use of the semantic information and association information of text features between different channels, and the traditional word vector representation method uses static method to extract text information, ignoring the location information of the text, which leads to inaccurate text sentiment classification, the Chinese short text sentiment classification model ALBERT-AFSAN, which is based on the combination of the attention feature split fusion network and the best, is proposed. Firstly, the model uses ALBERT to represent the word vector of the text to improve the representation ability of the word vector. Secondly, the feature is divided into two groups through the attention feature split fusion network, and then the features of the two groups of different channels are extracted and fused to retain the semantic association information between different channels to the greatest extent. Finally, the sentiment of Chinese short text is classified by softmax function, getting the emotional tendency of the text. The results show that the accuracy of the proposed method is 93.33%, 88.98% and 97.81% respectively on the three public datasets containing chnenticorp, waimai-10k and weibo-100k, and the F1 values are 93.23%, 88.47% and 97.78%, respectively. The results show that the proposed method can achieve better classification effect in sentiment analysis of Chinese short text.

Key words: ALBERT, split attention, feature fusion, sentiment analysis

叶星鑫, 徐杨, 罗梦诗. 基于ALBERT-AFSFN的中文短文本情感分析[J]. 计算机工程与应用, 2022, 58(12): 170-176.

YE Xingxin, XU Yang, LUO Mengshi. ALBERT-AFSFN Based Sentiment Analysis of Chinese Short Text[J]. Computer Engineering and Applications, 2022, 58(12): 170-176.

参考文献

[1] KUMAR R，VADLAMANI R.A survey on opinion mining and sentiment analysis：tasks，approaches and applications[J].Knowledge-Based Systems，2015，89：14-46.
[2] 廖小琴，徐杨.面向BSP-CNN的短文本情感倾向性分类研究[J].计算机工程与应用，2019，55（23）：120-124.
LIAO X Q，XU Y.Research on classification of short text emotional tendency for BSP-CNN[J].Computer Engineering and Applications，2019，55（23）：120-124.
[3] 朱晓霞，宋嘉欣，张晓缇.基于主题挖掘技术的文本情感分析综述[J].情报理论与实践，2019，42（11）：156-163.
ZHU X X，SONG J X，ZHANG X T，Review of text emotion analysis based on topic mining technology[J].Information Theory and Practice，2019，42（11）：156-163.
[4] 温超东，曾诚，任俊伟，等.结合ALBERT和双向门控循环单元的专利文本分类[J].计算机应用，2021，41（2）：407-412.
WEN C D，ZENG C，REN J W，et al.Patent text classification based on ALBERT and bidirectional gated recurrent unit[J].Journal of Computer Applications，2021，41（2）：407-412.
[5] MIKOLOV T，CHEN K，CORRADO G，et al.Efficient estimation of word representations in vector space[J].arXiv：1301.3781，2013.
[6] MIKOLOV T，SUTSKEVER I，CHEN K，et al.Distributed representations of words and phrases and their compositionality[C]//Advances in Neual Infomation Processing Systems，Lake Tahoe，Dec 5-10，2013.Spain：NIPS，2013：3111-3119.
[7] PENNINGTON J，SOCHER R，MANNING C D.Glove：global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing，Doha，Oct 25-29，2014.Qatar：EMNLP，2014：1532-1543.
[8] PETERS M E，NEUMANN M，IYYER M，et al.Deep contextualized word representations[J].arXiv：1802.05365，2018.
[9] RADFORD A，NARASIMHAN K，SALIMANS T，et al.Improving language understanding by generative pre-training[J/OL].（2018-06）[2020-11-01].https：//s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[10] DEVLIN J，CHANG M W，LEE K，et al.Bert：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[11] LAN Z，CHEN M，GOODMAN S，et al.Albert：a lite bert for self-supervised learning of language representations[J].arXiv：1909.11942，2019.
[12] 王志涛，於志文，郭斌，等.基于词典和规则集的中文微博情感分析[J].计算机工程与应用，2015，51（8）：218-225.
WANG Z T，YU Z W，GUO B，et al.Sentiment analysis of Chinese micro blog based on lexicon and rule set[J].Computer Engineering and Applications，2015，51（8）：218-225.
[13] 吴杰胜，陆奎.基于多部情感词典和规则集的中文微博情感分析研究[J].计算机应用与软件，2019，36（9）：93-99.
WU J S，LU K.Chinese weibo sentiment analysis based on multiple sentiment lexicons and rule sets[J].Computer Applications and Software，2019，36（9）：93-99.
[14] 黄进，阮彤，蒋锐权.基于SVM结合依存句法的金融领域舆情分析[J].计算机工程与应用，2015，51（23）：230-235.
HUANG J，RUAN T，JIANG R Q.Sentiment analysis in financial domain based on SVM with dependency syntax[J].Computer Engineering and Applications，2015，51（23）：230-235.
[15] 邓君，孙绍丹，王阮，等.基于Word2Vec和SVM的微博舆情情感演化分析[J].情报理论与实践，2020，43（8）：112-119.
DENG J，SUN S D，WANG R，et al.Evolution analysis of weibo public opinion emotion based on Word2Vec and SVM[J].Information Theory and Practice，2020，43（8）：112-119.
[16] MIKOLOV T，KOMBRINK S，BURGET L，et al.Extensions of recurrent neural network language model[C]//Proceedings of the 2011 International Conference on Acoustics，Speech and Signal Processing，Prague，May 22-27，2011.Piscataway：IEEE，2011：5528-5531.
[17] KIM Y.Convolutional neural networks for sentence classification[J].arXiv：1408.5882，2014.
[18] ZHOU P，SHI W，TIAN J，et al.Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics，Berlin，Germany，2016：207-212.
[19] ZHANG H，WU C，ZHANG Z，et al.Resnest：split-attention networks[J].arXiv：2004.08955，2020.
[20] DAI Y，GIESEKE F，OEHMCKE S，et al.Attentional feature fusion[J].arXiv：2009.14082，2020.