关键词: 室内场景语义分割, 深度学习, 注意力机制, 特征融合, 多尺度联合训练

Abstract:

Aiming at the problem that the current indoor scene semantic segmentation method cannot well integrate the RGB information and depth information of the image, an improved indoor scene semantic segmentation network is proposed. In order to enable the model to selectively fuse the depth features and RGB features of the image, and introduce the idea of attention mechanism, a feature fusion module is designed. According to the characteristics of depth feature map and RGB feature map, the module can adjust network parameters learning, and more effectively carry out deep fusion of depth features and RGB features. At the same time, multi-scale joint training is used to accelerate network convergence and improve segmentation accuracy. Through the verification on the SUNRGB-D and NYUDV2 datasets, compared to the current mainstream semantic segmentation networks such as RGB-D Fully Convolutional Neural Network（DFCN） with a Depth-sensitive fully-connected Conditional Random Field（DCRF）, Depth-aware convolutional neural networks （Depth-aware CNN）, Multi-path Refinement Network （RefineNet）, etc., the proposed network has higher segmentation accuracy, Mean Intersection over Union （mIoU） reached 46.6% and 48.0%, respectively.

Key words: indoor scene semantic segmentation, deep learning, attention mechanism, feature fusion, multi-scale joint training