Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (12): 122-131.DOI: 10.3778/j.issn.1002-8331.2107-0441

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Application of Spatial Temporal Graph Convolutional Networks in Human Abnormal Behavior Recognition

ZHANG Weilan, QI Hua, LI Sheng   

  1. 1.School of Electronic and Information Engineering, Xi’an Technological University, Xi’an 710021, China
    2.School of Information Engineering, Nanchang Institute of Technology, Nanchang 330200, China
  • Online:2022-06-15 Published:2022-06-15



  1. 1.西安工业大学 电子信息工程学院,西安 710021
    2.南昌工程学院 信息工程学院,南昌 330200

Abstract: Under the monitoring scenario, due to the shortage of monitoring resources, abnormal pedestrian behavior is prone to miss detection. In view of this problem, a method to identify abnormal human behavior in video surveillance scene is proposed to assist monitoring personnel to find abnormal behavior in time. The human skeleton is extracted from the pedestrians in the image using OpenPose. The graph attention mechanism based on the graph attention network (GAT) is integrated to solve the problem that the graph convolution network has a single way to aggregate the features of joint points. On the basis of the improved graph convolutional network, the spatial temporal graph convolutional networks(ST-GCN) is used to identify the abnormal behavior of pedestrians joint points information. The experimental results show that the recognition algorithm proposed in this paper has an accuracy of  85.48% for the defined behavior, and can accurately identify the abnormal behavior of pedestrians in surveillance video.

Key words: OpenPose algorithm, spatial temporal graph convolutional networks(ST-GCN), graph attention mechanism,  , behavior recognition

摘要: 在监控场景下,由于监控资源短缺,行人异常行为容易发生漏检。针对该问题,提出了一种视频监控场景下的人体异常行为识别的方法,辅助监控人员及时发现异常。使用OpenPose对图像中行人进行人体骨架提取。针对图卷积网络对关节点特征聚合方式单一的问题,融合了基于图注意力网络(graph attention network,GAT)的图注意力机制。在改进后的图卷积网络的基础上,利用时空图卷积神经网络(spatial temporal graph convolutional networks,ST-GCN),对行人关节点信息进行异常行为识别。实验结果表明,提出的识别算法对定义的行为识别准确率达85.48%,能够准确地识别监控视频中行人的异常行为。

关键词: OpenPose算法, 时空图卷积网络, 图注意力机制, 行为识别