Angle Space Adversarial Attack on Skeletal Action Recognition

doi:10.3778/j.issn.1002-8331.2203-0549

Abstract

Abstract: Deep learning models achieve good performance in skeleton-based action recognition tasks. However, the robustness of skeleton action recognition models has recently been questioned because they are vulnerable to adversarial attacks. At present, the existing white-box attack methods for skeleton action recognition cannot strictly constrain the unique spatial structure of the adversarial skeleton action samples, and the generated adversarial skeleton action samples are non-manifold adversarial samples, that is, the adversarial distribution is far from the original data distribution, which makes the skeleton action unnatural and easy to be perceived by humans. This paper proposes a new angular space adversarial attack method SCA（spherical coordinate attack） that uses spherical coordinate system to represent the skeleton structure. The experimental results on public datasets show that SCA can find most of the adversarial samples in the manifold space, while the existing white-box attack methods for skeleton action recognition can only find the adversarial samples in the non-manifold space.

Key words: adversarial attack, skeleton-based action recognition, deep learning, spherical coordinates, manifold adversarial examples

摘要： 深度学习模型在基于骨架数据的行为识别任务中取得了较好的性能。然而，骨架行为识别模型的鲁棒性最近受到了质疑，因为它们很容易受到对抗攻击的威胁。目前现有的行为识别白盒攻击方法无法对对抗骨架动作样本的独特空间结构进行严格约束，其生成的对抗骨架动作样本为非流形对抗样本，即对抗分布远离了原有的数据分布，使得骨架动作不自然，很容易被人眼察觉。提出了一种利用球坐标系表示骨架结构的角空间对抗攻击方法，球坐标攻击（spherical coordinate attack，SCA）。在公开数据集上的实验结果表明，SCA可以发现大部分存在流形空间上的对抗样本，而目前已有的行为识别白盒攻击方法只能寻找到非流形空间上的对抗样本。

关键词: 对抗攻击, 骨架行为识别, 深度学习, 球坐标, 流形对抗样本

CAO Nan, DIAO Yunfeng, HUANG Yinqin, DU Run, LI Huaixian, CHENG Tianjian. Angle Space Adversarial Attack on Skeletal Action Recognition[J]. Computer Engineering and Applications, 2023, 59(14): 260-267.

曹男, 刁云峰, 黄垠钦, 杜润, 李怀仙, 程天健. 面向骨架行为识别的角空间对抗攻击方法[J]. 计算机工程与应用, 2023, 59(14): 260-267.

References

[1] SZEGEDY C，ZAREMBA W，SUTSKEVER I，et al.Intriguing properties of neural networks[J].arXiv：1312.6199，2013.
[2] AKHTAR N，MIAN A.Threat of adversarial attacks on deep learning in computer vision：a survey[J].IEEE Access，2018，6：14410-14430.
[3] KARIM F，MAJUMDAR S，DARABI H.Adversarial attacks on time series[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，43（10）：3309-3320.
[4] FAWAZ H I，FORESTIER G，WEBER J，et al.Adversarial attacks on deep neural networks for time series classification[C]//2019 International Joint Conference on Neural Networks（IJCNN），2019：1-8.
[5] REN B，LIU M，DING R，et al.A survey on 3D skeleton-based action recognition using learning method[J].arXiv：2002.05907，2020.
[6] LIU J，AKHTAR N，MIAN A.Adversarial attack on skeleton-based human action recognition[J].IEEE Transactions on Neural Networks and Learning Systems，2022，33（4）：1609-1622.
[7] ZHENG T，LIU S，CHEN C，et al.Towards understanding the adversarial vulnerability of skeleton-based action recog-
nition[J].arXiv：2005.07151，2020.
[8] WANG H，HE F，PENG Z，et al.SMART：skeletal motion action recognition attack[J].arXiv：1911.07107，2019.
[9] SHAHROUDY A，LIU J，NG T T，et al.NTU RGB+D：a large scale dataset for 3D human activity analysis[C]//IEEE Conference on Computer Vision and Pattern Recognition，2016：1010-1019.
[10] DIAO Y，SHAO T，YANG Y L，et al.BASAR：black-box attack on skeletal action recognition[J].arxiv：2103.05266，2021.
[11] YAN S，XIONG Y，LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Thirty-Second AAAI Conference on Artificial Intelligence，2018.
[12] LIU Z，ZHANG H，CHEN Z，et al.Disentangling and unifying graph convolutions for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：143-152.
[13] CHEN Y，ZHANG Z，YUAN C，et al.Channel-wise topology refinement graph convolution for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2021：13359-13368.
[14] VEMULAPALLI R，ARRATE F，CHELLAPPA R.Human action recognition by representing 3D skeletons as points in a lie group[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2014：588-595.
[15] FERNANDO B，GAVVES E，ORAMAS J，et al.Modeling video evolution for action recognition[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：5378-5387.
[16] DEVANNE M，WANNOUS H，BERRETTI S，et al.3D human action recognition by shape analysis of motion trajectories on Riemannian manifold[J].IEEE Transactions on Cybernetics，2015，45（7）：1340-1352.
[17] DU Y，WANG W，WANG L.Hierarchical recurrent neural network for skeleton based action recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2015：1110-1118.
[18] LIU J，SHAHROUDY A，DONG X，et al.Spatio-temporal LSTM with trust gates for 3D human action recognition[C]//Computer Vision-ECCV，2016：816-833.
[19] SONG S，LAN C，XING J，et al.An end-to-end spatio-temporal attention model for human action recognition from skeleton data[C]//Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence，2017：4263-4270.
[20] ZHANG P，LAN C，XING J，et al.View adaptive neural networks for high performance skeleton-based human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，41（8）：1963-1978.
[21] LIU M Y，LIU H，CHEN C.Enhanced skeleton visuali-
zation for view invariant human action recognition[J].Pattern Recognition，2017，68：346-362.
[22] KE Q，BENNAMOUN M，AN S，et al.A new representation of skeleton sequences for 3D action recognition[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：4570-4579.
[23] KIM T S，REITER A.Interpretable 3D human action analysis with temporal convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops（CVPRW），2017：1623-1631.
[24] LI M，CHEN S，CHEN X，et al.Actional-structural graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：3595-3603.
[25] SHI L，ZHANG Y，CHENG J，et al.Two-stream adaptive graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：12026-12035.
[26] SHI L，ZHANG Y，CHENG J，et al.Skeleton-based action recognition with directed graph neural networks[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：7912-7921.
[27] CHENG K，ZHANG Y，HE X，et al.Skeleton-based action recognition with shift graph convolutional network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：183-192.
[28] ZHANG X，XU C，TAO D.Context aware graph convolution for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：14333-14342.
[29] GOODFELLOW I J，SHLENS J，SZEGEDY C.Explaining and harnessing adversarial examples[J].arXiv：1412.6572，2014.
[30] KURAKIN A，GOODFELLOW I，BENGIO S.Adversarial examples in the physical world[C]//International Conference on Learning Representations，2017.
[31] BAERLOCHER P.Inverse kinematics techniques of the interactive posture control of articulated figures[D].Lausanne：EPFL，2001.
[32] WANG H，HO E S L，KOMURA T.An energy-driven motion planning method for two distant postures[J].IEEE Transactions on Visualization and Computer Graphics，2014，21（1）：18-30.
[33] WANG H，HO E S L，SHUM H P H，et al.Spatio-temporal manifold learning for human motions via long-horizon modeling[J].IEEE Transactions on Visualization and Computer Graphics，2021，27（1）：216-227.
[34] CUI Q，SUN H，YANG F.Learning dynamic relationships for 3D human motion prediction[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：6519-6527.
[35] ATHALYE A，CARLINI N，WAGNER D.Obfuscated gradients give a false sense of security：circumventing defenses to adversarial examples[C]//International Conference on Machine Learning，2018：274-283.
[36] WANG H，DIAO Y，TAN Z，et al.Defending black-box skeleton-based human activity classifiers[J].arXiv：2203.04713，2022.