计算机工程与应用 ›› 2024, Vol. 60 ›› Issue (15): 101-110.DOI: 10.3778/j.issn.1002-8331.2304-0255

• 模式识别与人工智能 • 上一篇    下一篇

融合非负正弦位置编码和混合注意力机制的情感分析模型

郑志超,陈进东,张健   

  1. 1.北京信息科技大学 计算机学院,北京 100192
    2.北京信息科技大学 经济管理学院,北京 100192
    3.智能决策与大数据应用北京市国际科技合作基地,北京 100192
  • 出版日期:2024-08-01 发布日期:2024-07-30

Sentiment Classification Model Based on Non-Negative Sinusoidal Positional Encoding and Hybrid Attention Mechanism

ZHENG Zhichao, CHEN Jindong, ZHANG Jian   

  1. 1.School of Computer Science, Beijing Information Science and Technology University, Beijing 100192, China
    2.School of Economics and Management, Beijing Information Science and Technology University, Beijing 100192, China
    3.Beijing International Science and Technology Cooperation Base of Intelligent Decision and Big Data Application, Beijing 100192, China
  • Online:2024-08-01 Published:2024-07-30

摘要: 针对情感分析任务中,序列模型存在难以获取文本的相对位置信息,且处理较长序列时容易丢失关键信息等问题,提出了一种融合非负正弦位置编码(non-negative sinusoidal position encoding,NSPE)和混合注意力机制(hybrid attention mechanism,HAM)的双向长短期记忆网络(bi-directional long short-term memory,Bi-LSTM)情感分析模型(NSPEHA-BiLSTM)。提出NSPE方法,建立词语的NSPE,为词向量融入相对位置信息;通过Bi-LSTM提取文本特征,并基于HAM分别对特征的全局和局部特征进行赋权,确保关键信息的准确传递;通过全连接层实现文本情感分析。在IMDB数据集中,NSPEA-BiLSTM相较于Bi-LSTM和Text-CNN准确率分别提升了4.67和2.02个百分点,且输入的文本长度越长,模型效果越好,同时验证了NSPE优于其他位置编码。

关键词: 情感分析, 双向长短期记忆网络(Bi-LSTM), 非负正弦位置编码(NSPE), 混合注意力机制(HAM)

Abstract: NSPEHA-BiLSTM is proposed to address the issues of sequence models in sentiment analysis tasks, such as difficulty in obtaining the relative positional information of text and the loss of critical information when processing long sequences. The model integrates non-negative sinusoidal position encoding (NSPE) and hybrid attention mechanism (HAM) to incorporate relative positional information into word embeddings and weight the global and local information features of text using HAM, respectively, ensuring the accurate transmission of critical information. The text features are extracted by Bi-LSTM, and sentiment analysis is performed using a fully connected layer. NSPEHA-BiLSTM achieves higher accuracy than Bi-LSTM and Text-CNN by 4.67 and 2.02 percentage points, respectively, on the IMDB dataset, and the model performance improves with longer input text. The results also verify that NSPE is superior to other position encodings.

Key words: sentiment analysis, bi-directional long short-term memory (Bi-LSTM), non-negative sinusoidal position encoding (NSPE), hybrid attention mechanism (HAM)