Video Anomaly Detection Combining Memory-Augmented

doi:10.3778/j.issn.1002-8331.2012-0340

Abstract

Abstract: Deep autoencoder can predict the current frame to find abnormal events in videos. However, the autoencoder can not predict the low-level features of the image well, adding short-cut in the autoencoder can improve the ability to predict the details of the image. Autoencoder has good generalization ability, in order to suppress the accurate prediction of abnormal events, a memory augmented module is added between the short-cut of encoder and decoder to limit the accurate prediction of abnormal frames. At the same time, in order to highlight the events in the abnormal frame, the background of the current frame is obtained by the background extraction module while the video frame is predicted, which is used for the calculation of prediction error. The experimental results on UCSD Ped2 dataset, CUHK Avenue dataset and ShanghaiTech dataset show that the ability of anomaly detection is improved.

Key words: anomaly detection, memory-augmented, skip connection, auto encoder

摘要： 深度自编码器可以通过预测当前帧来判断视频中的异常情况。但由于自动编码器对图片的低层次特征无法良好的预测，在自动编码器中添加跳跃连接可以提高预测图片细节信息的能力。由于自动编码器有很好的“泛化”能力，为了抑制对异常事件的准确预测，通过在编码器和解码器的跳跃连接之间添加记忆增强模块限制模型对异常帧的准确预测。同时，为了突出异常帧中的事件，在预测视频帧的同时通过背景提取模块获取当前图片的背景信息用于后续预测误差的计算。在UCSD Ped2数据集、CUHK Avenue数据集和ShanghaiTech数据集上的实验结果表明，改进后模型的异常检测能力得到了提升。

关键词: 异常检测, 记忆增强, 跳跃连接, 自动编码器

CHEN Chen, HU Yan. Video Anomaly Detection Combining Memory-Augmented[J]. Computer Engineering and Applications, 2022, 58(15): 253-259.

陈澄, 胡燕. 融合记忆增强的视频异常检测[J]. 计算机工程与应用, 2022, 58(15): 253-259.

References

[1] LUO W，LIU W，GAO S.A revisit of sparse coding based anomaly detection in stacked RNN framework[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：341-349.
[2] LIU W，LUO W，LIAN D，et al.Future frame prediction for anomaly detection—a new baseline[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：6536-6545.
[3] LI Y，CAI Y，LIU J，et al.Spatio-temporal unity networking for video anomaly detection[J].IEEE Access，2019，7：172425-172432.
[4] 蔡瑞初，谢伟浩，郝志峰，等.基于多尺度时间递归神经网络的人群异常检测[J].软件学报，2015，26（11）：2884-2896.
CAI R C，XIE W H，HAO Z F，et al.Abnormal crowd detection based on multi-scale recurrent neural network[J].Journal of Software，2015，26（11）：2884-2896.
[5] 桂雅骏，吴小培，张超，等.融合音频和视频的室内智能监控系统[J].计算机工程与应用，2017，53（1）：220-226.
GUI Y J，WU X P，ZHANG C，et al.Indoor intelligent monitoring system with fusion of audio and video[J].Computer Engineering and Applications，2017，53（1）：220-226.
[6] HASAN M，CHOI J，NEUMANN J，et al.Learning temporal regularity in video sequences[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：733-742.
[7] GONG D，LIU L，LE V，et al.Memorizing normality to detect anomaly：memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proceedings of the IEEE International Conference on Computer Vision，2019：1705-1714.
[8] KIRAN B R，THOMAS D M，PARAKKAL R.An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos[J].Journal of Imaging，2018，4（2）：36.
[9] FENG Y，YUAN Y，LU X.Learning deep event models for crowd anomaly detection[J].Neurocomputing，2017，219：548-556.
[10] CONG Y，YUAN J，LIU J.Sparse reconstruction cost for abnormal event detection[C]//2011 IEEE Conference on Computer Vision and Pattern Recognition，2011：3449-3456.
[11] HASAN M，CHOI J，NEUMANN J，et al.Learning temporal regularity in video sequences[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：733-742.
[12] LAI Y，LIU R，HAN Y.Video anomaly detection via predictive autoencoder with gradient-based attention[C]//2020 IEEE International Conference on Multimedia and Expo（ICME），2020：1-6.
[13] YE M，PENG X，GAN W，et al.Anopcn：video anomaly detection via deep predictive coding network[C]//Proceedings of the 27th ACM International Conference on Multimedia，2019：1805-1813.
[14] RONNEBERGER O，FISCHER P，BROX T.U-net：Convolutional networks for biomedical image segmentation[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2015：234-241.
[15] CHAN A B，LIANG Z S J，VASCONCELOS N.Privacy preserving crowd monitoring：counting people without people models or tracking[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition，2008：1-7.
[16] LU C，SHI J，JIA J.Abnormal event detection at 150 fps in matlab[C]//Proceedings of the IEEE International Conference on Computer Vision，2013：2720-2727.
[17] MATHIEU M，COUPRIE C，LECUN Y.Deep multi-scale video prediction beyond mean square error[J].arXiv：1511.05440，2015.
[18] WANG L，XIONG Y，WANG Z，et al.Temporal segment networks for action recognition in videos[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，41（11）：2740-2755.
[19] ISOLA P，ZHU J Y，ZHOU T，et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1125-1134.
[20] GUPTA P，SRIVASTAVA P，BHARDWAJ S，et al.A modified PSNR metric based on HVS for quality assessment of color images[C]//2011 International Conference on Communication and Industrial Application，2011：1-4.
[21] LUO W，LIU W，GAO S.Remembering history with convolutional LSTM for anomaly detection[C]//2017 IEEE International Conference on Multimedia and Expo（ICME），2017：439-444.
[22] ZHAO Y，DENG B，SHEN C，et al.Spatio-temporal autoencoder for video anomaly detection[C]//Proceedings of the 25th ACM International Conference on Multimedia，2017：1933-1941.