Gait Recognition Combined with Convolutional Neural Network with Attention and Part-Level Features

doi:10.3778/j.issn.1002-8331.2009-0332

Abstract

Abstract: Currently, though deep learning algorithms have been widely used in the field of gait recognition, most existing methods extract global gait feature by convolutional neural networks, which ignores many local features with key gait information and to a certain extent, weakens the accuracy and improvement potential of gait recognition. To solve the above problems, a cross-view gait recognition method combined with convolutional neural network with attention and part-level features is proposed. By using gait silouettes as input, each gait frame passes through the same convolutional neural network with attention and forms the overall information, adding an effective attention mechanism CBAM into the network can show the importance of modeling the spaces and channels and increase the weight of significant regional features. Then the whole information is horizontally divided into two parts for training and gait recognition, the extracted gait local features are more suitable for fine gait classification. Finally, cross-view gait recognition experiments on the gait datasets CASIA-B and OU-ISIR-MVLP are carried out. The results show that the proposed method has a significant improvement effect compared with the existing methods under the condition of reorganization and insufficiency of the training dataset.

Key words: gait recognition, convolutional neural network, attention mechanism, part-level feature

摘要： 目前深度学习算法已经广泛应用于步态识别领域，但是大多数现有方法通过卷积神经网络提取步态全局特征时，忽略了许多包含关键步态信息的局部特征，在一定程度上削弱了步态识别的精度和提升潜力。针对上述问题，提出了一种结合注意力卷积神经网络与分块特征的跨视角步态识别方法，该方法以步态轮廓图序列为输入，每帧图片分别经过相同结构的注意力卷积神经网络融合成整体特征，在网络中加入有效的注意力机制CBAM能显式地建模各空间及通道的重要程度，增大显著区域特征的权重；整体特征被水平分成两块进行训练和步态识别，提取的步态局部特征更适合精细的步态分类。在步态数据集CASIA-B和OU-ISIR-MVLP上进行跨视角步态识别实验，结果证明在训练数据集充足与不足的条件下，该方法在识别精度上均优于现有方法。

关键词: 步态识别, 卷积神经网络, 注意力机制, 分块特征

HU Shaohui, WANG Xiuhui. Gait Recognition Combined with Convolutional Neural Network with Attention and Part-Level Features[J]. Computer Engineering and Applications, 2022, 58(3): 266-273.

胡少晖, 王修晖. 结合注意力卷积网络与分块特征的步态识别[J]. 计算机工程与应用, 2022, 58(3): 266-273.

References

[1] 贲晛烨.基于人体运动分析的步态识别算法研究[D].哈尔滨：哈尔滨工程大学，2010.
BEN X Y.Research on gait recognition algorithm based on human motion analysis[D].Harbin：Harbin Engineering University，2010.
[2] HAN J，BHANU B.Individual recognition using gait energy image[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2006，28（2）：316-322.
[3] JEEVAN M，JAIN N，HANMANDLU M，et al.Gait recognition based on gait pal and pal entropy image[C]//IEEE International Conference on Image Processing，2013.
[4] 王修晖，严珂.基于连续密度隐马尔可夫模型的人体步态识别[J].模式识别与人工智能，2016，29（8）：709-716.
WANG X H，YAN K.Human gait recognition using continuous density hidden Markov models[J].Pattern Recognition & Artificial Intelligence，2016，29（8）：709-716.
[5] MORE S A，DEORE P J.Gait recognition by cross wavelet transform and graph model[J].IEEE/CAA Journal of Automatica Sinica，2018，5（3）：718-726.
[6] 马勤勇，王申康，聂栋栋，等.基于瞬时步态能量图的远距离身份识别[J].电子学报，2007，35（11）：2078-2082.
MA Q Y，WANG S K，NIE D D，et al.Moment gait energy image based human recogition at a distance[J].Acta Electronica Sinica，2007，35（11）：2078-2082.
[7] 鲁继文，张二虎，薛延学.基于独立成分分析和信息融合的步态识别[J].模式识别与人工智能，2007，20（3）：365-370.
LU J W，ZHANG E H，XUE Y X.Gait recognition based on independent component analysis and information fusion[J].Pattern Recognition & Artificial Intelligence，2007，20（3）：365-370.
[8] WANG X H，WANG J，YAN K.Gait recognition based on gabor wavelets and（2D）2PCA[J].Multimedia Tools and Applications，2018，77（10）：12545-12561.
[9] CHEN Q，WANG Y H，LIU Z，et al.Feature map pooling for cross-view gait recognition based on silhouette sequence images[C]//International Geoscience and Remote Sensing Symposium，2018.
[10] WOLF T，BABAEE M，RIGOLL G.Multi-view gait recognition using 3D convolutional neural networks[C]//IEEE International Conference on Image Processing.Phoenix，AZ，the United states：IEEE，2016：4165-4169.
[11] SHIRAGA K，MAKIHARA Y，MURAMATSU D，et al.GEINet：view-invariant gait recognition using a convolutional neural network[C]//International Conference on Biometrics，2016.
[12] CHAO L，XIN M，LIN W Q，et al.DeepGait：a learning deep convolutional representation for view-invariant gait recognition using joint bayesian[J].Applied Sciences，2017，7（3）：210.
[13] TAKEMURA N，MAKIHARA Y，MURAMATSU D，et al.On input/output architectures for convolutional neural network-based cross-view gait recognition[J].IEEE Transactions on Circuits and Systems for Video Technology，2018，28（1）：1-13.
[14] ZHANG C，LIU W，MA H D，et al.Siamese neural network based gait recognition for human identification[C]//IEEE International Conference on Acoustics，2016.
[15] TONG S，FU Y，YUE X，et al.Multi-view gait recognition based on a spatial-temporal deep neural network[J]. IEEE Access，2018，6：57583-57596.
[16] WU Z F，HUANG Y Z，WANG L，et al.A comprehensive study on cross-view gait based human identification with deep CNNs[J].IEEE Transations on Pattern Analysis & Machine Intelligence，2017，39（2）：209-226.
[17] SANGHYUN W，JONGCHAN P，JOON-YOUNG L，et al.CBAM：convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision，2018：3-19.
[18] HU J，SHEN L，SUN G.Squeeze-and-excitation networks[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2018.
[19] SIMONYAN K，ZISSERMAN A.Very deep convolutional networks for large-scale image recognition[C]//Proceedings of the International Conference on Learning Representations，San Diego，USA，2015：1-14.
[20] KUSAKUNNIRAN W，WU Q，ZHANG J，et al.Support vector regression for multi-view gait recognition based on local motion feature selection[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition，2010：974-981.
[21] KUSAKUNNIRAN W，WU Q，ZHANG J，et al.Recognizing gaits across views through correlated motion co-clustering[J].IEEE Transactions on Image Processing，2014，23（2）：696-709.
[22] HU M D，WANG Y D，ZHANG Z X，et al.View-invariant discriminative projection for multi-view gait-based human identification[J].IEEE Transactions on Information Forensics and Security，2013，8（12）：2034-2045.
[23] PADOLE C，PROENCA H.An aperiodic feature representation for gait recognition in cross-view scenarios for unconstrained biometrics[J].Pattern Analysis and Applications，2017，20（1）：73-86.
[24] LI S Q，LIU W，MA H D.Attentive spatial-temporal summary networks for feature learning in irregular gait recognition[J].IEEE Transactions on Multimedia，2019，21（9）：2361-2375.
[25] YU S Q，WANG Q，SHEN L L，et al.View invariant gait recognition using only one uniform model[C]//Proceedings of the 23rd International Conference on Pattern Recognition，Cancun，2016：889-894.
[26] YU S Q，CHEN H F，REYES E B G，et al.GaitGAN：invariant gait feature extraction using generative adversarial network[C]//Conference on Computer Vision and Pattern Recognition Workshops.Honolulu：HI，2017：532-539.
[27] HE Y W，ZHANG J P，SHAN H M，et al.Multi-task GANs for view-specific feature learning in gait recognition[J].IEEE Transactions on Information Forensics and Security，2019，14（1）：102-113.