Abstract: The reconstruction model based on generative adversarial networks may correspond to small reconstruction errors when reconstructing video frames. In addition, adversarial training in prediction and reconstruction models is often unstable, which affects the detection performance of the model. To address the above issues, a bidirectional prediction network using both predictor and discriminator is proposed based on generative adversarial networks to detect abnormal human behavior in videos. The training process of this network is divided into two stages. The first stage extracts the temporal and spatial information of the input video frames through a predictor, and introduces an attention mechanism to focus on the actual motion area. It predicts the intermediate frames of the normal video frame sequence while preserving the state of the predictor during the training process. In the second stage, the role of the discriminator is changed from distinguishing between generated data and real data to distinguishing the quality of predicted frames. The discriminator learns to detect subtle distortions that often occur when generating abnormal input predicted frames, improving the stability of the training process and the accuracy of the detection results. The model achieves frame level AUC of 98.7%, 91.8%, and 84.6% on the UCSD Ped2, Avenue, and ShanghaiTech datasets used for video human abnormal behavior detection.

Key words: generative adversarial networks, human abnormal behavior, predictor, discriminator

摘要： 基于生成对抗网络的重构模型在重构视频帧时可能对应较小的重构误差，此外预测和重构模型中的对抗性训练往往是不稳定的，影响模型的检测性能。针对上述问题，基于生成对抗网络提出一种同时使用预测器和判别器的双向预测网络检测视频中的人体异常行为。该网络的训练过程分为两个阶段：第一个阶段通过预测器提取输入视频帧的时间和空间信息，并引入注意力机制关注实际发生运动的区域，对正常视频帧序列的中间帧进行预测，同时保留训练过程中预测器的状态；第二阶段将判别器的角色从区分生成数据和真实数据改为区分预测帧质量的高低，判别器学会检测在生成异常输入的预测帧时经常出现的细微失真，提高了训练过程的稳定性和检测结果的准确性。该模型在用于视频人体异常行为检测的UCSD Ped2、Avenue和ShanghaiTech三个数据集上，帧级别AUC分别达到了98.7%、91.8%、84.6%。

关键词: 生成对抗网络, 人体异常行为, 预测器, 判别器