Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (23): 1-14.DOI: 10.3778/j.issn.1002-8331.2303-0254

• Research Hotspots and Reviews • Previous Articles     Next Articles

Review of Multimodal Sensor Data Fusion in Sentiment Analysis

JIN Yelei, Gulanbaier Tuerhong, Mairidan Wushouer   

  1. School of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
  • Online:2023-12-01 Published:2023-12-01

情感分析中的多传感器数据融合研究综述

金叶磊,古兰拜尔·吐尔洪,买日旦·吾守尔   

  1. 新疆大学 信息科学与工程学院,乌鲁木齐 830046

Abstract: Sentiment analysis technology in the context of multi-sensor data fusion is a hot research direction in the field of human-computer interaction. With the advancement of deep learning techniques, research on sentiment analysis has shifted from traditional approaches based on single-sensor data to methods that leverage the fusion of multiple sensor data. This paper aims to provide an overview of the research status and challenges of sentiment analysis technology under the framework of multi-sensor data fusion, starting from the definition and development process of multi-sensor data fusion and sentiment analysis technology. It introduces the classic models and traditional methods of multi-sensor data fusion,and discusses the main research directions and achievements in sentiment analysis, including studies related to sentiment analysis based on various data modalities such as speech, visual, textual, and physiological information. Lastly, it presents multimodal sentiment analysis methods based on multi-sensor data fusion, compares the performance of multimodal sentiment analysis with single-modal approaches through experiments, and explores the future prospects and possible research directions of sentiment analysis technology under the framework of multi-sensor data fusion, including cross-lingual sentiment analysis and further applications and advancements of multimodal sentiment analysis technology.

Key words: sensor, data fusion, emotion recognition, multimode, feature fusion

摘要: 多传感器数据融合技术下的情感分析技术是人机交互领域的热门研究方向。随着深度学习技术的发展,情感分析技术的研究也从传统的基于单一传感器数据的方法转向了基于多传感器数据融合的方法。从多传感器数据融合以及情感分析技术的定义和发展历程出发,介绍多传感器数据融合技术下的情感分析技术的研究现状和挑战;介绍了多传感器数据融合的经典模型和传统方法。阐述了目前国内外情感分析技术研究的主要方向和研究成果,其中包括基于如语音、视觉、文本和生理信息等数据的情感分析相关研究。介绍了基于多传感器数据融合技术的多模态情感分析方法,通过实验对比多模态情感分析与单模态的情感分类效果,并展望了多传感器数据融合技术下的情感分析技术的发展前景和可能的研究方向,其中包括跨语种情感分析及多模态情感分析技术的进一步应用和发展等。

关键词: 传感器, 数据融合, 情感分析, 多模态, 特征融合