Spatio-Temporal Fusion Gait Recognition Method Combining Silhouette and Pose

doi:10.3778/j.issn.1002-8331.2204-0500

Abstract

Abstract: Most of the existing gait recognition methods are contour-based gait recognition methods, however, contours are easily affected by occlusion, resulting in a decrease in recognition accuracy. In real monitoring scenarios, occlusion is almost inevitable, and improving the accuracy of gait recognition under occlusion is the premise that the algorithm can land in practical applications. Aiming at this problem, a spatio-temporal fusion gait recognition method combining silhouette and pose is proposed. Using the ability of pose to resist occlusion, a multi-modality spatial feature fusion module is designed, and the feature reuse strategy and modal fusion strategy are used to improve the information capacity of spatial features. A multi-scale temporal feature extraction module is designed to extract temporal information at different time scales using independent branches, and an attention-based feature fusion strategy is proposed to integrate temporal information adaptively. A spatial feature set branch is designed to improve the representation of spatial-temporal features in a deeply supervised manner. Experimental results on publicly available datasets show the effectiveness of the proposed method, and the model has good robustness under occlusion.

Key words: gait recognition, occlusion resistance, multi-modality, multi-scale, attention

摘要： 现有的大多数步态识别方法是基于轮廓的步态识别方法，然而轮廓容易受到遮挡的影响，从而导致识别准确率下降。在现实的监控场景下，遮挡几乎是不可避免的，提高遮挡情况下的步态识别精度是算法能够“落地”于实际应用的前提。针对此问题，提出了结合轮廓与姿态的时空融合步态识别方法。利用姿态具有抵抗遮挡的能力，设计多模态空间特征融合模块，利用特征重用策略和模态融合策略以提高空间特征的信息容量；设计多尺度时间特征提取模块，利用独立分支提取不同时间尺度下的时间信息，提出一种基于注意力的特征融合策略以自适应地整合时间信息；设计空间特征集合分支，以深监督方式提高时空特征的表达能力。在公开数据集上的实验结果表明了所提方法的有效性，模型在遮挡情况下具有较好的鲁棒性。

关键词: 步态识别, 抵抗遮挡, 多模态, 多尺度, 注意力

ZHANG Chaoyue, ZHANG Rong. Spatio-Temporal Fusion Gait Recognition Method Combining Silhouette and Pose[J]. Computer Engineering and Applications, 2023, 59(16): 135-142.

张超越, 张荣. 结合轮廓与姿态的时空融合步态识别方法[J]. 计算机工程与应用, 2023, 59(16): 135-142.

References

[1] CHAO H，WANG K，HE Y，et al.GaitSet：cross-view gait recognition through utilizing gait as a deep set[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2022，44（7）：3467-3478.
[2] FAN C，PENG Y，CAO C，et al.GaitPart：temporal part-based model for gait recognition[C]//Proceedings of the 2000 IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：14225-14233.
[3] HOU S，LIU X，CAO C，et al.Set residual network for silhouette-based gait recognition[J].IEEE Transactions on Biometrics Behavior and Identity Science，2021，3（3）：384-393.
[4] IWAMA H，MURAMATSU D，MAKIHARA Y，et al.Gait verification system for criminal investigation[J].IPSJ Transactions on Computer Vision and Applications，2013，5：163-175.
[5] LYNNERUP N，LARSEN P K.Gait as evidence[J].IET Biometrics，2014，3（2）：47-54.
[6] ZMA B，HFM A，IB B，et al.Investigating the use of motion-based features from optical flow for gait recognition-ScienceDirect[J].Neurocomputing，2018，283：140-149.
[7] ZHANG Y，HUANG Y，YU S，et al.Cross-view gait recognition by discriminative feature learning[J].IEEE Transactions on Image Processing，2019，29：1001-1015.
[8] LIAO R，CAO C，GARCIA E B，et al.Pose-based temporal-spatial network （PTSN） for gait recognition with carrying and clothing variations[C]//Proceedings of the 12th Chinese Conference on Biometric Recognition.Cham：Springer，2017：474-483.
[9] TEEPE T，KHAN A，GILG J，et al.GaitGraph：graph convolutional network for skeleton-based gait recognition[C]//Proceedings of the 2021 IEEE International Conference on Image Processing，2021：2314-2318.
[10] YU S，TAN D，TAN T.A framework for evaluating the effect of view angle，clothing and carrying condition on gait recognition[C]//Proceedings of the 18th International Conference on Pattern Recognition，2006，4：441-444.
[11] ZHU Z，GUO X，YANG T，et al.Gait recognition in the wild：a benchmark[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision，2021：14789-14799.
[12] WEI S E，RAMAKRISHNA V，KANADE T，et al.Convolutional pose machines[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，2016：4724-4732.
[13] TAKEMURA N，MAKIHARA Y，MURAMATSU D，et al.Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition[J].IPSJ Transactions on Computer Vision & Applications，2018，10（1）：4.
[14] HERMANS A，BEYER L，LEIBE B.In defense of the triplet loss for person re-identification[J].arXiv：1703.07737，2017.
[15] HUANG G，LIU Z，VAN DER MAATEN L，et al.Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：4700-4708.
[16] JIE H，LI S，GANG S，et al.Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[17] LEE C Y，XIE S，GALLAGHER P，et al.Deeply-supervised nets[C]//Proceedings of the 18th International Conference on Artificial Intelligence and Statistics，2014：562-570.
[18] JU H，BHANU B.Individual recognition using gait energy image[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2005，28（2）：316-322.