关键词: 手势特征, 操作动作, 视频特征提取, 动作识别

Abstract:

In view of manipulation action recognition in dynamic and complex scenes, an action recognition framework based on gesture feature fusion is proposed. The framework mainly contains an RGB video feature extraction module, a gesture feature extraction module and an action classification module. The RGB video feature extraction module mainly uses the I3D network to extract the temporal and spatial features of the RGB videos; the gesture feature extraction module uses the Mask R-CNN network to extract the operator’s gesture features; the action classification module merges the above features and inputs them into a classifier for classification. On the EPIC-Kitchens dataset, the accuracy of the proposed method for grasp gestures recognition is 89.63%, and the accuracy of recognizing comprehensive actions reaches 74.67%.

Key words: gesture feature, manipulation action, video feature extraction, action recognition