Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (16): 135-142.DOI: 10.3778/j.issn.1002-8331.2204-0500

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Spatio-Temporal Fusion Gait Recognition Method Combining Silhouette and Pose

ZHANG Chaoyue, ZHANG Rong   

  1. Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, Zhejiang 315211, China
  • Online:2023-08-15 Published:2023-08-15

结合轮廓与姿态的时空融合步态识别方法

张超越,张荣   

  1. 宁波大学 信息科学与工程学院,浙江 宁波 315211

Abstract: Most of the existing gait recognition methods are contour-based gait recognition methods, however, contours are easily affected by occlusion, resulting in a decrease in recognition accuracy. In real monitoring scenarios, occlusion is almost inevitable, and improving the accuracy of gait recognition under occlusion is the premise that the algorithm can land in practical applications. Aiming at this problem, a spatio-temporal fusion gait recognition method combining silhouette and pose is proposed. Using the ability of pose to resist occlusion, a multi-modality spatial feature fusion module is designed, and the feature reuse strategy and modal fusion strategy are used to improve the information capacity of spatial features. A multi-scale temporal feature extraction module is designed to extract temporal information at different time scales using independent branches, and an attention-based feature fusion strategy is proposed to integrate temporal information adaptively. A spatial feature set branch is designed to improve the representation of spatial-temporal features in a deeply supervised manner. Experimental results on publicly available datasets show the effectiveness of the proposed method, and the model has good robustness under occlusion.

Key words: gait recognition, occlusion resistance, multi-modality, multi-scale, attention

摘要: 现有的大多数步态识别方法是基于轮廓的步态识别方法,然而轮廓容易受到遮挡的影响,从而导致识别准确率下降。在现实的监控场景下,遮挡几乎是不可避免的,提高遮挡情况下的步态识别精度是算法能够“落地”于实际应用的前提。针对此问题,提出了结合轮廓与姿态的时空融合步态识别方法。利用姿态具有抵抗遮挡的能力,设计多模态空间特征融合模块,利用特征重用策略和模态融合策略以提高空间特征的信息容量;设计多尺度时间特征提取模块,利用独立分支提取不同时间尺度下的时间信息,提出一种基于注意力的特征融合策略以自适应地整合时间信息;设计空间特征集合分支,以深监督方式提高时空特征的表达能力。在公开数据集上的实验结果表明了所提方法的有效性,模型在遮挡情况下具有较好的鲁棒性。

关键词: 步态识别, 抵抗遮挡, 多模态, 多尺度, 注意力