Implicit Sentiment Classification Model Based on Enhancement of Sentiment Features Oriented to Chinese Text

doi:10.3778/j.issn.1002-8331.2209-0026

Abstract

Abstract: The semantic features in the implicit sentiment sentence cannot be deeply mined because the lack of explicit sentiment words, which inevitably affects the classification accuracy. To solve the problem, this paper proposes a implicit sentiment classification model based on the enhancement of sentiment features oriented to Chinese text, named CISC. To improve the classification accuracy, the positive and negative sentiment lexicons are constructed, and the sentiment words are embedded into the position to get sentences with an enhancement of the sentiment features. Firstly, the sentences are preprocessed to get the corresponding word sequence. Then, the sentiment words are positioned and embedded with positive and negative words respectively through self-attention. The corresponding positive and negative sentence representations are obtained respectively through hierarchical attention networks. Next, the corresponding sentence representations are input into the Bi-GRU models and AOA to get corresponding feature vectors respectively. Finally, the obtained feature vectors are input into the Softmax to get the sentiment tendency. Further, the positive sentiment probability is calculated for each sentence incorporating positive words. Similarly, the negative sentiment probability can be gotten for each sentence incorporating negative words. So the final sentiment tendency is gotten by comparing the average value between the positive and the negative sentiment probability. Experiments on the SMP-ECISA2019 public dataset show that the proposed model of CISC has a higher classification performance of Chinese implicit sentiment text compared to EBA, GGBA models.

Key words: implicit sentiment, sentiment classification, sentiment word, feature enhancement, semantic feature

摘要： 隐式情感句子由于缺乏显式情感词并且其语义特征无法深入挖掘，导致现有模型进行情感分类时准确率不高。针对这一现状，提出一种基于情感特征增强的中文隐式情感分类模型（CISC），通过构建积极和消极情感词库，并将情感词进行位置嵌入得到情感特征增强的句子，进而提高分类准确率。对句子进行预处理得到对应的词语序列；通过自注意力机制的情感词检测方法进行句子情感词定位并分别嵌入积极和消极词，借助多层注意力网络得到对应的正向和负向的句子表示；分别将获取到的句子表示通过Bi-GRU模型和交互注意力机制（attention over attention，AOA）提取出对应的语义特征；将语义特征分别通过Softmax进行情感倾向概率计算，通过融入积极词的句子正向情感概率与融入消极词句子的负向情感概率进行均值计算并比较，得到最终的情感倾向。与EBA、GGBA等多种模型在SMP-ECISA2019公开数据集进行了比对实验，实验结果证明，提出的CISC模型可以提高中文隐式情感文本的分类效果。

关键词: 隐式情感, 情感分类, 情感词, 特征增强, 语义特征

TAN Guangpu, ZHU Guangli, WEI Siyu. Implicit Sentiment Classification Model Based on Enhancement of Sentiment Features Oriented to Chinese Text[J]. Computer Engineering and Applications, 2024, 60(3): 196-204.

谈光璞, 朱广丽, 韦斯羽. 基于情感特征增强的中文隐式情感分类模型[J]. 计算机工程与应用, 2024, 60(3): 196-204.

References

[1] DAUPHIN Y N, FAN A, AULI M, et al. Language modeling with gated convolutional networks[C]//Proceedings of the 34th International Conference on Machine Learning. New York: JMLR. org, 2017: 933-941.
[2] ZHANG Y, WALLACE B. A sensitivity analysis of (and practitioners’ guide to) convolutional neural networks for sentence classification[C]//Proceedings of the 8th International Joint Conference on Natural Language Processing.[S.l.]: Asian Federation of Natural Language Processing, 2017: 253-263.
[3] MA Y, CHENG G Q, LIANG X X, et al. Heterogeneous graph neural networks based on meta-path[C]//Proceeding of 2020 3rd International Conference on Algorithms, Computing and Artificial Intelligence (ACAI 2020), 2020: 95-98.
[4] YAO L, MAO C, LUO Y. Graph convolutional networks for text classification[C]//The Thirty-Third AAAI Conference on Artificial Intelligence(AAAI-19), 2019: 7370-7377.
[5] CHEN J, HOU H, JI Y, et al. Graph convolutional networks with structural attention model for aspect based sentiment analysis[C]//International Joint Conference on Neural Networks, Budapest, Hungary, 2019: 1-7.
[6] WANG B X. Disconnected recurrent neural networks for text categorization[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2018: 2311-2320.
[7] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: Association for Computational Linguistics, 2019: 4171-4186.
[8] SUN Y, WANG S H, LI Y K , et al. ERNIE2.0: a continual pretraining framework for language understanding[C]//Proceedings of AAAI Conference on Artificial Intelligence, New York, NY, USA, 2020: 8968-8975.
[9] WANG H B , HOU M H , LI F, et al. Chinese implicit sentiment analysis based on hierarchical knowledge enhancement and multi-pooling[J]. IEEE Access, 2020, 8: 126051-126065.
[10] ZHUANG Y, LIU Z , LIU T T, et al. Implicit sentiment analysis based on multi-feature neural network model[J]. Soft Computing, 2022, 26(2): 635-644.
[11] SUN Z Q, YU H K, SONG X D, et al. MobileBERT: a compact task-agnostic BERT for resource-limited devices[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: Association for Computational Linguistics, 2020: 2158-2170.
[12] KNYAZEV B, GRAHAM W T, MOHAMED R A. Understanding attention and generalization in graph neural networks[C]//33rd Conference on Neural Information Processing Systems(NeurIPS 2019), Vancouver, Canada, 2019.
[13] XIANG C L, REN Y F , JI D H. Identifying implicit polarity of events by using an attention-based neural network model[J]. IEEE Access, 2019, 7: 133170-133177.
[14] 赵容梅, 熊熙, 琚生根, 等. 基于混合神经网络的中文隐式情感分析[J]. 四川大学学报(自然科学版), 2020, 57(2): 264-270.
ZHAO R M , XIONG X, JU S G , et al. Implicit sentiment analysis for Chinese texts based on a hybird netural netork[J]. Journal of Sichuan University (Natural Science Edition), 2020, 57(2): 264-270.
[15] HE R, LEE W S, NG H T, et al. An unsupervised neural attention model for aspect extraction[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics ( Volume 1: Long Papers), 2017: 388-397.
[16] LIAO J , WANG S G , LI D Y. Identification of fact-implied implicit sentiment based on multi-level semantic fused representation[J]. Knowledge-Based Systems, 2019, 165(1): 197-207.
[17] 郭凤羽, 贺瑞芳, 党建武. 基于语境交互感知和模式筛选的隐式篇章关系识别[J]. 计算机学报, 2020, 43(5): 901-915.
GUO F Y , HE R F , DANG J W. Implicit discourse relation recognition based on contextual interaction perception and pattern filtering[J]. Chinese Journal of Computers, 2020, 43(5): 901-915.
[18] RANA T A, CHEAH Y N, RANA T. Multi-level knowledge-based approach for implicit spect identification[J]. Applied Intelligence, 2020, 50(12): 4616-4630.
[19] ZHANG Y R , XIE Y X , SUN J Y. Aspect level sentiment classification based on viewpoint information unit[C]//2021 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA), Dalian, 2021: 40-46.
[20] WEI J Y, LIAO J, YANG Z F , et al. BiLSTM with multi-polarity orthogonal attention for implicit sentiment analysis[J]. Neurocomputing, 2020, 383: 165-173.
[21] ZHAO N, GAO H, WEN X, et al. Combination of convolutional neural network and gated recurrent unit for aspect-based sentiment analysis[J]. IEEE Access, 2021, 9: 15561-15569.
[22] 袁景凌, 丁远远, 潘东行, 等. 基于时序和上下文特征的中文隐式情感分类模型[J]. 计算机应用, 2021, 41(10): 2820-2828.
YUAN J L , DING Y Y, PAN D X, et al. Chinese implicit sentiment classification model based on sequence and contextual features[J]. Computer Application, 2021, 41(10): 2820-2828.
[23] 黄山成, 韩东红, 乔百友, 等. 基于ERNIE2.0-BiLSTM-Attention的隐式情感分析方法[J]. 小型微型计算机系统, 2021, 42(12): 2485-2489.
HUANG S C , HAN D H , QIAO B Y , et al. Implicit sentiment analysis method based on ERNIE2.0-BiLSTM-Attention[J]. Journal of Chinese Computer Systems, 2021, 42(12): 2485-2489.
[24] 陈秋嫦, 赵晖, 左恩光, 等. 上下文感知的树递归神经网络下隐式情感分析[J]. 计算机工程与应用, 2022, 58(7): 167-175.
CHEN Q C , ZHAO H , ZUO E G , et al. Implicit sentiment analysis based on context aware tree recurrent neutral network[J]. Computer Engineering and Applications, 2022, 58(7): 167-175.
[25] YANG Z, YANG D, DYER C, et al. Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2016: 1480-1489.
[26] 杨善良, 常征. 基于图注意力神经网络的中文隐式情感分析[J]. 计算机工程与应用, 2021, 57(24): 161-167.
YANG S L, CHANG Z. Chinese implicit sentiment analysis based on graph attention neural network[J]. Computer Engineering and Applications, 2021, 57(24): 161-167.
[27] 潘东行, 袁景凌, 李琳, 等. 一种融合上下文特征的中文隐式情感分类模型[J]. 计算机工程与科学, 2020, 42(2): 341-350.
PAN D X, YUAN J L, LI L, et al. A Chinese implicit sentiment classification model combining contextual features[J]. Computer Engineering and Science, 2020, 42(2): 341-350.