Abstract: This paper proposes an view-invariant feature extraction method based on multi-view non-negative matrix factorization to fuse multi-view information and recognize human activity. The spatio-temporal descriptors of each video frame are extracted to describe motion and morphological information in the video scene. In order to solve the effect of observation angle change on recognition, spatio-temporal matrix based on spatio-temporal descriptors is constructed from different perspectives, and multi-view non-negative matrix factorization is used to construct multi-view objective functions to obtain a consensus matrix with multi-view information. The behavior is classified by calculating the maximum correlation coefficient of the consensus matrix. This method is validated on WVU dataset and i3Dpose dataset, and compared with other methods. The results show that the method is effective in behavior recognition.

Key words: multi-view non-negative matrix factorization, spatio-temporal matrix, consensus matrix, maximum correlation coefficient

摘要： 提出一种基于多视角非负矩阵分解的视角不变特征提取方法用于融合多视角信息并进行人体行为识别。通过提取每个视频帧的时空描述符，有效描述了视频场景中的运动和形态信息；为了解决观测角度改变对识别的影响，在不同视角下构建基于时空描述符的时空矩阵，并利用多视角非负矩阵分解构建多视角的目标函数以得到融合了多视角信息的共识矩阵；计算共识矩阵的最大相关系数进行人体行为分类。该方法在WVU数据集、i3Dpose数据集上进行了验证，并与其他方法进行比较，结果表明了该方法在行为识别方面的有效性。

关键词: 多视角非负矩阵分解, 时空矩阵, 共识矩阵, 最大相关系数