Combining Reconstruction and Graph Prediction for Multi-Dimensional Time Series Anomaly Detection Framework

doi:10.3778/j.issn.1002-8331.2304-0073

Abstract

Abstract: High-dimensional time series anomaly detection has been a significant challenge in the field of intelligent system security. Existing mainstream solutions often rely on data dimensionality reduction for reconstruction or time series modeling for prediction. However, these methods lack the integration of inter-feature interactions and intra-feature temporal dependencies, and mostly utilize point estimation, thereby compromising the accuracy of anomaly detection. This paper proposes a novel end-to-end anomaly detection framework that combines the advantages of prediction and reconstruction, considering the overall distribution of sequences. Firstly, an improved variational autoencoder reconstruction module is designed to learn intra-feature temporal dependencies and obtain a low-dimensional encoded representation. Secondly, a graph neural network prediction module estimates Gaussian distributions, integrating the low-dimensional representation from the reconstruction module and the original input for capturing inter-feature structural dependencies. Lastly, the model employs an anomaly scoring module that combines the losses of the reconstruction and prediction modules, enabling spatiotemporal joint representation while considering the overall distribution of sequences. To validate the performance of the proposed model, comparative experiments are conducted on three industrial datasets, demonstrating its superior performance in terms of the F1 metric compared to baseline models.

Key words: multivariate time series data, graph neural network, autoencoder, anomaly detection

摘要： 高维时序异常检测一直是智能系统安全领域的重要挑战，主流解决方案通常使用基于数据降维的重构方法和基于时序建模的预测方法，但这些方法没有结合特征间相互影响和特征内时间关联进行学习，且大多使用点估计方法进行预测或重构，从而影响了异常检测的准确性。结合预测和重构的优点，考虑序列的整体分布，提出了一种新颖的端到端异常检测框架。设计改进的变分自动编码器重构模块，以学习原始时序数据中的特征内时间关联，同时得到编码后的低维表示。设计估计高斯分布的图神经网络预测模块，结合重构模块的低维表示和原始输入进行图结构学习，以捕捉特征间的结构依赖。模型采用异常评分模块联合重构和预测模块的损失，在考虑序列整体分布的基础上进行时空联合表征。为验证所提出模型的性能，在三个工业数据集上对模型进行了对比实验，与基线模型相比，所提出的模型在F1性能指标上表现良好。

关键词: 多元时序数据, 图神经网络, 自编码器, 异常检测

WU Yanwen, TAN Xichen, GE Di, HAN Yuan, XIONG Xujie, CHEN Yudi. Combining Reconstruction and Graph Prediction for Multi-Dimensional Time Series Anomaly Detection Framework[J]. Computer Engineering and Applications, 2024, 60(13): 301-310.

吴彦文, 谭溪晨, 葛迪, 韩园, 熊栩捷, 陈宇迪. 结合重构和图预测的多元时序异常检测框架[J]. 计算机工程与应用, 2024, 60(13): 301-310.

References

[1] FENG C, TIAN P. Time series anomaly detection for cyber-physical systems via neural system identification and Bayesian filtering[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 2021.
[2] WU H S. A survey of research on anomaly detection for time series[C]//Proceedings of the 13th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), 2016.
[3] COOK A A, MISIRLI G, FAN Z. Anomaly detection for IoT time-series data: a survey[J]. IEEE Internet of Things Journal, 2020, 7(7): 6481-6494.
[4] AUDIBERT J, MICHIARDI P, GUYARD F, et al. Do deep neural networks contribute to multivariate time series anomaly detection?[J]. Pattern Recogntion, 2022: 132: 108945.
[5] 张锦, 程良伦. 一种分布式CPS异常检测的无监督图模型[J]. 计算机工程与应用, 2018, 54(12): 110-115.
ZHANG J, CHENG L L. Unsupervised graphical model for anomaly detection in distributed CPS[J]. Computer Engineering and Applications, 2018, 54(12): 110-115.
[6] DARBAN Z. Z, WEBB G. I, PAN S, et al. Deep learning for time series anomaly detection: a survey[J]. arXiv:2211.05244, 2022.
[7] LINDEMANN B, MüLLER T, VIETZ H, et al. A survey on long short-term memory networks for time series prediction[J]. Procedia CIRP, 2021, 99: 650-655.
[8] JIN M, KOH H Y, WEN Q, et al. A survey on graph neural networks for time series: forecasting, classification, imputation, and anomaly detection[J]. arXiv:2307.03759, 2023.
[9] KIM H, LEE B, SHIN W Y, et al. Graph anomaly detection with graph neural networks: current status and challenges[J]. IEEE Access, 2022, 10: 111820-111829.
[10] YU Q, LYU J, JIANG L. An improved ARIMA-based traffic anomaly detection algorithm for wireless sensor networks[J]. International Journal of Distributed Sensor Networks, 2016, 12(1): 9653230.
[11] MELNYK I, MATTHEWS B, VALIZADEGAN H, et al. Vector autoregressive model-based anomaly detection in aviation systems[J]. Journal of Aerospace Information Systems, 2016: 13(4): 161-173.
[12] AMER M, GOLDSTEIN M, ABDENNADHER S. Enhancing one-class support vector machines for unsupervised anomaly detection[C]//Proceedings of the ACM SIGKDD Workshop on Outlier Detection and Description, 2013: 8-15.
[13] DENG H, RUNGER G, TUV E, et al. A time series forest for classification and feature extraction[J]. Information Sciences, 2013, 239: 142-153.
[14] CHOI K, YI J, PARK C, et al. Deep learning for anom aly detection in time-series data: review, analysis, and guidelines[J]. IEEE Access, 2021, 9: 120043-120065.
[15] 惠飞, 郭静, 贾硕, 等. 基于双向长短记忆网络的异常驾驶行为检测[J]. 计算机工程与应用, 2020, 56(24): 116-122.
HUI F, GUO J, JIA S, et al. Detection of abnormal driving behavior based on BiLSTM[J]. Computer Engineering and Applications, 2020, 56(24): 116-122.
[16] ZHANG H, XIA Y, YAN T, et al. Unsupervised anomaly detection in multivariate time series through transformer-based variational autoencoder[C]//Proceedings of the 33rd Chinese Control and Decision Conference (CCDC), 2021: 281-286.
[17] 安磊, 韩忠华, 林硕, 等. 面向网络入侵检测的GAN-SDAE-RF模型研究[J]. 计算机工程与应用, 2021, 57(21): 155-164.
AN L, HAN Z H, LIN S, et al. Research on GAN-SDAE-RF model for network intrusion detection[J]. Computer Engineering and Applications, 2021, 57(21): 155-164.
[18] LI D, CHEN D, JIN B, et al. MAD-GAN: multivariate anomaly detection for time series data with generative adversarial networks[C]//Proceedings of the International Conference on Artificial Neural Networks. Cham: Springer International Publishing, 2019: 703-716.
[19] PARK D, HOSHI Y, KEMP C C. A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder[J]. IEEE Robotics and Automation Letters, 2018, 3(3): 1544-1551.
[20] HAN S, WOO S S. Learning sparse latent graph representations for anomaly detection in multi-variate time series[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022: 2977-2986.
[21] ZHOU J, CUI G, HU S, et al. Graph neural networks: a review of methods and applications[J]. AI Open, 2020, 1: 57-81.
[22] KIPF T, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv:1609.02907, 2016.
[23] KIPF T, WELLING M. Variational graph auto-encoders[J]. arXiv:1611.07308, 2016.
[24] FENG Y, CHEN J, LIU Z, et al. Full graph autoencoder for one-class group anomaly detection of IIoT system[J]. IEEE Internet of Things Journal, 2022, 9(21): 21886-21898.
[25] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks[J]. arXiv:1710.10903, 2017.
[26] ZHAO H, WANG Y, DUAN J, et al. Multivariate time-series anomaly detection via graph attention network[C]//Proceedings of the IEEE International Conference on Data Mining (ICDM), 2020: 841-850.
[27] GUAN S, ZHAO B, DONG Z, et al. GTAD: graph and temporal neural network for multivariate time series anomaly detection[J]. Entropy, 2022, 24(6): 759.
[28] DENG A, HOOI B. Graph neural network-based anomaly detection in multivariate time series[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2022: 4027-4035.
[29] RAY S, LAKDAWALA S, GOSWAMI M, et al. Learning graph neural networks for multivariate time series anomaly detection[J]. arXiv:2111.08082, 2021.
[30] HE Z, GAO S, XIAO L, et al. Wider and deeper, cheaper and faster: tensorized LSTMs for sequence learning[C]//Neural Information Processing Systems, 2017: 1-11.
[31] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017: 5998-6008.
[32] SHYU M L, CHEN S C, SARINNAPAKORN K, et al. A novel anomaly detection scheme based on princpal component classifier[C]//Proceedings of the IEEE Foundations and New Directions of Data Mining Workshop, 2003: 172-179.
[33] ANGIULLI F, PIZZUTI C. Fast outlier detection in high dimensional spaces[C]//Proceedings of the European Conference on Principles of Data Mining and Knowledge Discovery, 2002: 15-27.
[34] SAKURADA M, YAIRI T. Anomaly detection using autoencoders with nonlinear dimensionality reduction[C]//Proceedings of the MLSDA 2nd Workshop on Machine Learning for Sensory Data Analysis, 2014 : 4-11.
[35] ZONG B, SONG Q, MIN M, et al. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection[C]//Proceedings of the International Conference on Learning Representations, 2018.