面向超像素块级记忆学习的视频异常检测

doi:10.3778/j.issn.1002-8331.2404-0251

摘要/Abstract

摘要： 针对当前视频异常检测模型中网络泛化能力过强，导致某些异常帧也能被很好预测的问题，提出一种面向超像素块级记忆学习的视频异常检测方法（superpixel block-level memory learning oriented video anomaly detection，SBM-VAD）。通过超像素分割算法SLIC调整特征图的分割粒度，以在实时原型记忆库（real-time prototype memory bank，RTPMB）中存储更精细的原型粒度，从而增强正常帧与异常帧在特征表示中的区分性。在U-Net预测网络的每层跳跃连接处引入一个与分割大小相匹配的实时原型记忆库，以避免模型学习输入到输出的恒等映射，有效约束传入解码器的特征。此外，采用一种基于软注意力策略的信号去噪模块（signal denoising module，SDM），引导模型优先学习预测前景区域，从而获得具有高质量前景的预测帧。在UCSD、CUHK Avenue和Shanghai Tech公开数据集上进行实验验证，结果表明改进后的模型在异常检测能力和效果方面均有显著提升。

关键词: 视频异常检测, 过泛化能力, 原型粒度, 恒等映射, 信号去噪

Abstract: Aiming at the problem that the generalization ability of the network is too strong in current video anomaly detection models, which leads to the accurate prediction of some abnormal frames, a method for superpixel block-level memory learning oriented video anomaly detection(SBM-VAD) is proposed. Firstly, by employing the superpixel segmentation algorithm SLIC to adjust the granularity of feature map segmentation, finer prototype granularity is stored in the real-time prototype memory bank (RTPMB), thereby enhancing the discriminative ability between normal frames and abnormal frames in feature representation. Secondly, a real-time prototype memory bank matching the segmentation size is introduced at each layer jump connection of the U-Net prediction network to avoid the model learning the input-to-output identity mapping and effectively constrain the incoming features to the decoder. In addition, a signal denoising module based on soft attention strategy is used to guide the model to preferentially learn to predict the foreground region, so as to obtain prediction frames with high-quality foreground. Finally, the experimental verification is carried out on UCSD, CUHK Avenue and ShanghaiTech public data sets, and the results show that the improved model has significantly improved the anomaly detection capability and effect.

Key words: video anomaly detection, overgeneralization ability, prototype granularity, identity mapping, signal denoising

谢斌红, 王乾, 张睿, 张英俊, 陆望东. 面向超像素块级记忆学习的视频异常检测[J]. 计算机工程与应用, 2025, 61(14): 297-306.

XIE Binhong, WANG Qian, ZHANG Rui, ZHANG Yingjun, LU Wangdong. Superpixel Block-Level Memory Learning Oriented Video Anomaly Detection[J]. Computer Engineering and Applications, 2025, 61(14): 297-306.

参考文献

[1] BENEZETH Y, JODOIN P M, SALIGRAMA V, et al. Abnormal events detection based on spatio-temporal co-occurences[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2009: 2458-2465.
[2] 陈澄, 胡燕. 融合记忆增强的视频异常检测[J]. 计算机工程与应用, 2022, 58(15): 253-259.
CHEN C, HU Y. Video anomaly detection combining memory-augmented[J]. Computer Engineering and Applications, 2022, 58(15): 253-259.
[3] ZHAO Y R, DENG B, SHEN C, et al. Spatio-temporal AutoEncoder for video anomaly detection[C]//Proceedings of the 25th ACM International Conference on Multimedia. New York: ACM, 2017: 1933-1941.
[4] RAVANBAKHSH M, NABI M, SANGINETO E, et al. Abnormal event detection in videos using generative adversarial nets[C]//Proceedings of the IEEE International Conference on Image Processing. Piscataway: IEEE, 2017: 1577-1581.
[5] GONG D, LIU L Q, LE V, et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 1705-1714.
[6] PARK H, NOH J, HAM B. Learning memory-guided normality for anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 14360-14369.
[7] LV H, CHEN C, CUI Z, et al. Learning normal dynamics in videos with meta prototype network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 15420-15429.
[8] LIU W R, CHANG H, MA B P, et al. Diversity-measurable anomaly detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 12147-12156.
[9] LIU W, LUO W X, LIAN D Z, et al. Future frame prediction for anomaly detection-a new baseline[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 6536-6545.
[10] HOU J L, ZHANG Y Y, ZHONG Q Y, et al. Divide-and-assemble: learning block-wise memory for unsupervised anomaly detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 8771-8780.
[11] ZHONG Y H, CHEN X, JIANG J Y, et al. Reverse erasure guided spatio-temporal autoencoder with compact feature representation for video anomaly detection[J]. Science China Information Sciences, 2022, 65(9): 194101.
[12] CHEN Y Q, ZHOU X S, HUANG T S. One-class SVM for learning in image retrieval[C]//Proceedings of the International Conference on Image Processing. Piscataway: IEEE, 2001: 34-37.
[13] TAX D M J, DUIN R P W. Support vector data description[J]. Machine Learning, 2004, 54(1): 45-66.
[14] ZIMEK A, SCHUBERT E, KRIEGEL H P. A survey on unsupervised outlier detection in high-dimensional numerical data[J]. Statistical Analysis and Data Mining: the ASA Data Science Journal, 2012, 5(5): 363-387.
[15] AKCAY S, ATAPOUR-ABARGHOUEI A, BRECKON T P. Skip-GANomaly: skip connected and adversarially trained encoder-decoder anomaly detection[C]//Proceedings of the International Joint Conference on Neural Networks. Piscataway: IEEE, 2019: 1-8.
[16] WANG X H, DU Y, LIN S J, et al. adVAE: a self-adversarial variational autoencoder with Gaussian anomaly prior knowledge for anomaly detection[J]. Knowledge-Based Systems, 2020, 190: 105187.
[17] HAO Y, LI J, WANG N N, et al. Spatiotemporal consistency-enhanced network for video anomaly detection[J]. Pattern Recognition, 2022, 121: 108232.
[18] 黄少年, 文沛然, 全琪, 等. 基于多支路聚合的帧预测轻量化视频异常检测[J]. 图学学报, 2023, 44(6): 1173-1182.
HUANG S N, WEN P R, QUAN Q, et al. Future frame prediction based on multi-branch aggregation for lightweight video anomaly detection[J]. Journal of Graphics, 2023, 44(6): 1173-1182.
[19] HE K M, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2980-2988.
[20] ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2282.
[21] VEDALDI A, SOATTO S. Quick-shift and kernel methods for mode seeking[C]//Proceedings of the European Conference on Computer Vision. Berlin, Heidelberg: Springer, 2008: 705-718.
[22] VEKSLER O, BOYKOV Y, MEHRANI P. Superpixels and supervoxels in an energy optimization framework[C]//Proceedings of the European Conference on Computer Vision. Berlin, Heidelberg: Springer, 2010: 211-224.
[23] BERGH A D M, BOIX X, ROIG G, et al. SEEDS: superpixels extracted via energy-driven sampling[C]//Proceedings of the European Conference on Computer Vision. Berlin, Heidelberg: Springer, 2012: 13-26.
[24] 宋熙煜, 周利莉, 李中国, 等. 图像分割中的超像素方法研究综述[J]. 中国图象图形学报, 2015, 20(5): 599-608.
SONG X Y, ZHOU L L, LI Z G, et al. Review on superpixel methods in image segmentation[J]. Journal of Image and Graphics, 2015, 20(5): 599-608.
[25] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[26] CAO Y , XU J , LIN S , et al. GCNet: non-local networks meet squeeze-excitation networks and beyond[J]. arXiv:1904.11492, 2019.
[27] LI W, MAHADEVAN V, VASCONCELOS N. Anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(1): 18-32.
[28] LU C W, SHI J P, JIA J Y. Abnormal event detection at 150 FPS in MATLAB[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE, 2013: 2720-2727.
[29] LUO W X, LIU W, GAO S H. A revisit of sparse coding based anomaly detection in stacked RNN framework[C]//Proceedings of the IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 341-349.
[30] LUO W X, LIU W, GAO S H. Remembering history with convolutional LSTM for anomaly detection[C]//Proceedings of the IEEE International Conference on Multimedia and Expo. Piscataway: IEEE, 2017: 439-444.
[31] HU C, ZHU L Q, LAI S X. Spatio-temporal-based context fusion for video anomaly detection[C]//Proceedings of the International Conference on Pattern Recognition, Machine Vision and Intelligent Algorithms. Piscataway: IEEE, 2023: 187-192.
[32] HUANG X Y, ZHAO C D, GAO C X, et al. Synthetic pseudo anomalies for unsupervised video anomaly detection: a simple yet efficient framework based on masked autoencoder[C]//Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2023: 1-5.
[33] WANG Y Z, QIN C, BAI Y, et al. Making reconstruction-based method great again for video anomaly detection[C]//Proceedings of the IEEE International Conference on Data Mining. Piscataway: IEEE, 2022: 1215-1220.
[34] HU C, WU F, WU W, et al. Normal learning in videos with attention prototype network[J]. arXiv:2108.11055, 2021.
[35] WANG Y, LIU T Y, ZHOU J G, et al. Video anomaly detection based on spatio-temporal relationships among objects[J]. Neurocomputing, 2023, 532: 141-151.
[36] 杨静, 吴成茂, 周流平. 基于全局-局部自注意力网络的视频异常检测方法[J]. 通信学报, 2023, 44(8): 241-250.
YANG J, WU C M, ZHOU L P. Novel video anomaly detection method based on global-local self-attention network[J]. Journal on Communications, 2023, 44(8): 241-250.
[37] WEN X P, LAI H C, GAO G X, et al. Video anomaly detection based on cross-frame prediction mechanism and spatio-temporal memory-enhanced pseudo-3D encoder[J]. Engineering Applications of Artificial Intelligence, 2023, 126: 107057.