Parallel Emotion Analysis Model Based on Text Abstract Extraction

doi:10.3778/j.issn.1002-8331.2205-0488

Abstract

Abstract: Aiming at the problems that the traditional text classification model has limited recognition ability and the training time doubles with the input length. A parallel feature extraction emotion analysis model BLAT based on text abstract extraction is proposed. BLAT model introduces additive attention mechanism of Fastformer to replace self attention mechanism of Transformer, so that the model can have an excellent training speed in the face of long text training without losing accuracy. Secondly, the model forms parallel features by extracting the original text data, and integrates the long-term and short-term memory network and convolutional neural network to form a parallel feature extraction network to realize the efficient recognition of text emotional tendency. It verifies on the Chinese e-commerce review data set, the accuracy can reach 92.26%, which can achieve better results compared with the current mainstream model.

Key words: abstract extraction, additive attention mechanism, feature fusion, emotion analysis

摘要： 针对传统文本分类模型存在识别能力受限、训练时间随着输入长度倍增的问题，提出了一种基于文本摘要提取的双路特征情感分析模型（BLAT）。BLAT模型引入Fastformer的加性注意力机制代替Transfomer的自注意力机制，使得模型能够在不损失精度的情况下，面对长文本训练能够有较为出色的训练速度。模型通过对原始文本数据做摘要提取处理形成双路特征，融入长短期记忆网络与卷积神经网络组成双路特征提取网络，实现对文本情感倾向的高效识别。通过实验在中文电商评论数据集上进行验证，准确率可以达到92.26%，相较当下主流模型能够达到较好的效果。

关键词: 摘要提取, 加性注意力机制, 特征融合, 情感分析

WANG Zhixiang, LIU Yuan. Parallel Emotion Analysis Model Based on Text Abstract Extraction[J]. Computer Engineering and Applications, 2023, 59(18): 119-128.

王郅翔, 刘渊. 基于文本摘要提取的双路情感分析模型[J]. 计算机工程与应用, 2023, 59(18): 119-128.

References

[1] 中国互联网络信息中心.第48次中国互联网络发展状况统计报告[EB/OL].[2021-09].https：//www.cnnic.cn/hlwfzyj/hlwxzbg/hlwtjbg/202109/P020210915523670981527.pdf.China Internet Network Information Center.48th meeting of statistical reports on internet development in China[EB/OL].[2021-09].https：//www.cnnic.cn/hlwfzyj/hlwxzbg/hlwtjbg/202109/P020210915523670981527.pdf.
[2] PANG B，LEE L.Opinion mining and sentiment analysis[J].Foundations and Trends in Information Retrieval，2008，2（1/2）：130-135.
[3] 汤凌燕，熊聪聪，王嫄，等.基于深度学习的短文本情感倾向分析综述[J].计算机科学与探索，2021，15（5）：794-811.
TANG L Y，XIONG C C，WANG Y，et al.Review of deep learning for short text sentiment tendency analysis[J].Journal of Frontiers of Computer Science and Technology，2021，15（5）：794-811.
[4] MIKOLOV T，SUTSKEVER I，CHEN K，et al.Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems.Cambridge：MIT Press，2013：3111-3119.
[5] PENNINGTON J，SOCHER R，MANNING C D.GloVe：global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing（EMNLP），2014：1532-1543.
[6] KIM Y.Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.Doha：EMNLP，2014：1746-1751.
[7] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems.Long Beach：NIPS，2017：6000-6010.
[8] WANG Y，HUANG M，ZHU X，et al.Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing，2016：606-615.
[9] 赵宏，傅兆阳，赵凡.基于BERT和层次化Attention的微博情感分析研究[J].计算机工程与应用，2022，58（5）：156-162.
ZHAO H，FU Z Y，ZHAO F.Microblog sentiment analysis based on BERT and hierarchical attention[J].Computer Engineering and Applications，2022，58（5）：156-162.
[10] 赵亚南，刘渊，宋设.融合多头自注意力机制的金融新闻极性分析[J].计算机工程，2020，46（8）：85-92.
ZHAO Y N，LIU Y，SONG S.Financial news polarity analysis fusing with multi-head self-attention mechanism[J].Computer Engineering，2020，46（8）：85-92.
[11] 齐甜方，蒋洪迅.基于Seq2Seq文本摘要和情感挖掘的股票波动趋势预测[J].管理评论，2021，33（5）：257-269.
QI T F，JIANG H X.Exploring stock price trend using Seq2Seq based automatic text summarization and senti-ment mining[J].Business Review，2021，33（5）：257-269.
[12] 李金鹏，张闯，陈小军，等.自动文本摘要研究综述[J].计算机研究与发展，2021，58（1）：1-21.
LI J P，ZHANG C，CHEN X J，et al.Survey on automatic text summarization[J].Journal of Computer Research and Development，2021，58（1）：1-21.
[13] DAI Z，YANG Z，YANG Y，et al.Transformer-xl：attentive language models beyond a fixed-length context[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence，Italy：Association for Computational Linguistics，2019：2978-2988.
[14] YANG Z，DAI Z，YANG Y，et al.Xlnet：generalized autoregressive pretraining for language understanding[C]//Annual Conference on Neural Information Processing Systems，Vancouver，BC，Canada，December 8-14，2019：5754-5764.
[15] WU C，WU F，QI T，et al.Fastformer：additive attention can be all you need[J].arXiv：2108.09084，2021.
[16] XIAO Z，LIANG P J.Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//International Conference on Cloud Computing and Security.Cham：Springer，2016：601-610.
[17] 田乔鑫，孔韦韦，滕金保，等.基于并行混合网络与注意力机制的文本情感分析模型[J].计算机工程，2022，48（8）：266-273.
TIAN Q X，KONG W W，TENG J B，et al.Text sentiment analysis model based on parallel hybrid network and attention mechanism[J].Computer Engineering，2022，48（8）：266-273.
[18] 万俊杰，任丽佳，单鸿涛，等.双通道的BCBLA情感分类模型[J].小型微型计算机系统，2023，44（5）：954-960.
WAN J J，REN L J，SHAN H T，et al.Two-channel sentiment classification model BCBLA[J].Journal of Chinese Computer Systems，2023，44（5）：954-960.
[19] LIU M，WANG L，NIE L.Weibo-oriented Chinese news summarization via multi-feature combination[C]//CCF International Conference on Natural Language Processing and Chinese Computing.Cham：Springer，2015：581-589.
[20] WANG X，HE X，NIE L，et al.Item silk road：recommending items from information domains to social users[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval，2017：185-194.