Research on Chinese Irony Recognition by Integrating BERT and Bidirectional Long Short-Term Memory Networks

doi:10.3778/j.issn.1002-8331.2307-0015

Abstract

Abstract: Users comment on hot topics on microblog using rhetorical techniques such as irony and sarcasm. Irony and sarcasm themselves carry a certain emotional tendency, which has a certain tendency to affect the sentiment analysis results. Therefore, this paper focuses on irony recognition of Chinese microblog comments, constructs a tri-classified dataset containing ironic, sarcasm and non-irony, and proposes a model BERT_BiLSTM based on bidirectional encoder representations from Transformers (BERT) and bidirectional long short-term memory network (BiLSTM). The model generates dynamic word vectors containing contextual information through BERT, inputs BiLSTM to extract the deep ironic features of the text, and passes in softmax at the fully connected layer for ironic recognition of the text. The experimental results indicate that the BERT_BiLSTM model proposed in this paper has significantly improved the accuracy and F1 values compared with the existing mainstream models on both binary and triple classification datasets.

Key words: irony recognition, bidirectional encoder representations from Transformers (BERT), feature extraction, bidirectional long short-term memory network (BiLSTM)

摘要： 用户对微博热点话题进行评论时会使用反语、讽刺的修辞手法，其本身带有一定的情感倾向会对情感分析结果造成一定影响。因此该文主要针对中文微博评论进行反讽识别，构建了一个包含反语、讽刺和非反讽的三分类数据集，提出一个基于BERT和双向长短时记忆网络（BiLSTM）的模型BERT_BiLSTM。该模型通过BERT生成含有上下文信息的动态字向量，输入BiLSTM提取文本的深层反讽特征，在全连接层传入softmax对文本进行反讽识别。实验结果表示，在二分类和三分类数据集上，提出的BERT_BiLSTM模型与现有主流模型相比准确率和F1值均有明显提高。

关键词: 反讽识别, BERT, 特征提取, 双向长短时记忆网络（BiLSTM）

WANG Xuyang, QI Nan, WEI Shenyou. Research on Chinese Irony Recognition by Integrating BERT and Bidirectional Long Short-Term Memory Networks[J]. Computer Engineering and Applications, 2024, 60(20): 153-159.

王旭阳, 戚楠, 魏申酉. 融合BERT和双向长短时记忆网络的中文反讽识别研究[J]. 计算机工程与应用, 2024, 60(20): 153-159.

References

[1] 陈婧扬. 新媒体舆论场主流媒体情绪传播管理与引导研究[D]. 上海: 上海大学, 2021.
CHEN J Y. Research on the management and guidance of mainstream media emotion communication in the new media public opinion field[D]. Shanghai: Shanghai University, 2021.
[2] DIETER D. Irony, disruption and moral imperfection[J]. Ethical Theory and Moral Practice, 2020, 23(3/4): 545-559.
[3] PENG W, ADIKARI A, ALAHAKOON D, et al. Discovering the influence of sarcasm in social media responses[J]. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2019, 9: 201129817.
[4] TIV M, ROUILLARD V, VINGRON N, et al. Global second language proficiency predicts self-perceptions of general sarcasm use among bilingual adults[J]. Journal of Language and Social Psychology, 2019, 38(4): 459-478.
[5] REN L, XU B, LIN H F, et al. Sarcasm detection with sentiment semantics enhanced multi-level memory network[J]. Neurocomputing, 2020, 401: 320-326.
[6] 缪亚林, 姬怡纯, 张顺, 等. CNN-BiGRU模型在中文短文本情感分析的应用[J]. 情报科学, 2021, 39(4): 85-91.
MIAO Y L, JI Y C, ZHANG S, et al. Application of CNN-BiGRU model in Chinese short text sentiment analysis[J]. Information Science, 2021, 39(4): 85-91.
[7] SON L H, KUMAR A, SANGWAN S R, et al. Sarcasm detection using soft attention-based bidirectional long short-term memory model with convolution network[J]. IEEE Access, 2019, 7: 23319-23328.
[8] 卢欣, 李旸, 王素格. 融合语言特征的卷积神经网络的反讽识别方法[J]. 中文信息学报, 2019, 33(5): 31-38.
LU X, LI Y, WANG S G. Linguistic features enhanced convolutional neural networks for irony recognition[J]. Journal of Chinese Information Processing, 2019, 33(5): 31-38.
[9] ZHANG S W, ZHANG X Z, ROSSO P. Irony detection via sentiment-based transfer learning[J]. Information Processing & Management, 2019, 56(5): 1633-1644.
[10] 杨善良, 常征. 基于图注意力神经网络的中文隐式情感分析[J]. 计算机工程与应用, 2021, 57(24): 161-167.
YANG S L, CHANG Z. Chinese implicit sentiment analysis based on graph attention neural network[J]. Computer Engineering and Applications, 2021, 57(24): 161-167.
[11] SAIT A R W, ISHAK M K. Deep learning with natural language processing enabled sentimental analysis on sarcasm classification[J]. Computer Systems Science and Engineering, 2023, 44(3): 2553-2567.
[12] DONG Y F, ZHANG Y T, LI J, et al. DC-BiGRU-CNN algorithm for irony recognition in Chinese social comments[J]. Mathematical Problems in Engineering, 2022(1): 1-14.
[13] 张军, 张丽, 沈凡凡, 等. RoBERTa融合BiLSTM及注意力机制的隐式情感分析[J]. 计算机工程与应用, 2022, 58(23): 142-150.
ZHANG J, ZHANG L, SHEN F F, et al. Implicit sentiment analysis based on RoBERTa fused with BiLSTM and attention mechanism[J]. Computer Engineering and Applications, 2022, 58(23): 142-150.
[14] 潘宏鹏, 汪东, 刘忠轶, 等. 考虑反讽语义识别的协同双向编码舆情评论情感分析研究[J]. 情报杂志, 2022, 41(5): 99-105.
PAN H P, WANG D, LIU Z Y, et al. Public opinion comments sentiment analysis research considering ironic semantic recognition based on the collaborative BERT[J]. Journal of Intelligence, 2022, 41(5): 99-105.
[15] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019: 4171-4186.
[16] JIANG X C, SONG C, XU Y C, et al. Research on sentiment classification for netizens based on the BERT-BiLSTM-TextCNN model[J]. PeerJ Computer Science, 2022, 8: 1005.
[17] ABDEL-SALAM S, RAFEA A. Performance study on extractive text summarization using BERT models[J]. Information, 2022, 13(2): 67.
[18] WU X Q, XIA Y, ZHU J H, et al. A study of BERT for context-aware neural machine translation[J]. Machine Learning, 2022, 111(3): 917-935.
[19] XU F, ZHANG X F, XIN Z H, et al. Investigation on the Chinese text sentiment analysis based on convolutional neural networks in deep learning[J]. Computers, Materials, & Continua, 2019, 58(3): 697-709.
[20] CHEN H L, LI SHI, WU P H, et al. Fine-grained sentiment analysis of Chinese reviews using LSTM network[J]. Journal of Engineering Science and Technology Review, 2018, 11(1): 174-179.
[21] CHENG Y Y, CHEN Y M, YEH W C, et al. Valence and arousal-infused bi-directional LSTM for sentiment analysis of government social media management[J]. Applied Sciences-Basel, 2021, 11(2): 1-14.
[22] JIANG M, ZHANG W, ZHANG M, et al. An LSTM-CNN attention approach for aspect-level sentiment classification[J]. Journal of Computational Methods in Sciences and Engineering, 2019, 19(4): 859-868.
[23] KUMAR S N K, MALARVIZHI N. Bi-directional LSTM-CNN combined method for sentiment analysis in part of speech tagging (PoS)[J]. International Journal of Speech Technology, 2020, 23(2): 373-380.