Double Feature Hierarchical Embedding Multivariate Time Series Anomaly Detection Method

doi:10.3778/j.issn.1002-8331.2307-0188

Abstract

Abstract: It is of great significance to carry out anomaly detection of real-time operation status of industrial entity equipment under multivariate time series characteristics for maintaining stable operation of complex industrial system and promoting national economic development, improving quality and efficiency. Aiming at the problem that the existing anomaly detection methods are insufficient in analyzing the characteristics of the highly nonlinear time dependence of the equipment running state and the various dimensional correlations among the monitoring data of the internal sensors, considering the situation that the distribution of monitoring data is unknown and the train data might be mixed with noise or abnormal data, a multivariate time series anomaly detection method based on double feature hierarchical embedding is proposed. The sequence time feature is compressed through time-gated recurrent units and the data distribution is expanded to obtain time embedded variables with the affine mechanism of the flow model to capture global and local features of long-term sequences. At the same time, the original input is mapped to the latent space using a variational autoencoder. Embedding time features through the flow model of shared parameters and further analyzing the coupling relationship between sequences based on gated cyclic units, fully mining the time dependence and dimension correlation of time series, improving abnormality detection accuracy. Experiment is carried out on 5 authoritative published multivariate time series datasets. Compared with 12 typical time series anomaly detection methods, the proposed algorithm improved in two evaluation indexes, and ranked first in the two indexes on average, which verifies the advanced nature and effectiveness of the proposed method.

Key words: multivariate time series anomaly detection, recurrent neural network, variational autoencoder, flow model, hierarchical feature embedding

摘要： 开展多维时序特征下的工业实体设备实时运行状态在线异常检测，对维护复杂工业系统稳定运行、推动国家经济发展提质增效具有重要意义。针对现有异常检测方法对时序数据高度非线性的时间依赖关系及其模式多样的维度耦合关系分析不足的问题，综合考虑监控数据分布未知导致训练数据中可能掺杂噪声或异常数据的情况，提出双特征层次嵌入的多维时序异常检测方法。通过循环神经网络对时序特征数据进行处理，引入流模型仿射机制拓展数据分布并得到时间嵌入变量，捕捉长时间序列的全局及局部特征；与此同时，利用变分自编码器将多维输入映射到潜空间，共享时间嵌入的流模型参数，基于门控循环单元对维度间的耦合关系进一步关联分析，充分挖掘多维时序数据的时间依赖性和维度相关性，提高异常检测准确率。在5个权威公开的多维时序数据集上开展实验，与12种典型时序异常检测方法进行对比，所提算法在多种评价指标上的平均排名均位列第一，验证了所提方法的先进性和有效性。

关键词: 多维时序异常检测, 循环神经网络, 变分自编码器, 流模型, 层次特征嵌入

CHEN Wenli, SU Yu, CHEN Lingli, GAO Xin, CHENG Yingying, ZOU Bo. Double Feature Hierarchical Embedding Multivariate Time Series Anomaly Detection Method[J]. Computer Engineering and Applications, 2024, 60(21): 142-153.

陈文礼, 苏宇, 陈玲俐, 高欣, 程瑛颖, 邹波. 双特征层次嵌入的多维时序异常检测方法[J]. 计算机工程与应用, 2024, 60(21): 142-153.

References

[1] 孙海丽, 龙翔, 韩兰胜, 等. 工业物联网异常检测技术综述[J]. 通信学报, 2022, 43(3): 196-210.
SUN H L, LONG X, HAN L S, et al. Overview of anomaly detection techniques for industrial Internet of things[J]. Journal on Communications, 2022, 43(3): 196-210.
[2] ZHANG G Y, GAO X, WANG L, et al. Probabilistic autoencoder with multi?scale feature extraction for multivariate time series anomaly detection[J]. Applied Intelligence, 2023, 53: 15855-15872.
[3] 宁剑, 任怡睿, 林济铿, 等. 基于人工智能及信息融合的电力系统故障诊断方法[J]. 电网技术, 2021, 45(8): 2925-2933.
NING J, REN Y R, LIN J K, et al. Power system fault diagnosis based on artificial intelligence and information fusion[J]. Power System Technology, 2021, 45(8): 2925-2933.
[4] 谢伟, 卢士达, 时宽治, 等. 面向工业物联网时序数据的异常检测方法[J]. 计算机工程与应用, 2024, 60(12): 270-282.
XIE W, LU S D, SHI K Z, et al. Anomaly detection method for industrial IoT timing data[J]. Computer Engineering and Applications , 2024, 60(12): 270-282.
[5] KIM S, CHOI K, CHOI H S, et al. Towards a rigorous evaluation of time-series anomaly detection[C]//Proceedingds of the 36th AAAI Conference on Artificial Intelligence, 2022: 7194-7201.
[6] “中国学科及前沿领域发展战略研究 (2021—2035)”项目组. 中国工业互联网2035发展战略[M]. 北京: 科学出版社, 2023: 193-203.
“Development Strategy of Discipline and Frontier Research in China (2021-2035)” Project Team. China industrial Internet 2035 development strategy[M]. Beijing: Science Press, 2023:193-203.
[7] AHMAD S, LAVIN A, PURDY S, et al. Unsupervised real-time anomaly detection for streaming data[J]. Neurocomputing, 2017, 262: 134-147.
[8] 应斐昊, 邢宁哲, 纪雨彤, 等. 基于KTLAD的电力数据网业务流量异常检测[J]. 北京邮电大学学报, 2017, 40(S1): 108-111.
YING F H, XING N Z, JI Y T, et al. KTLAD based traffic anomaly detection algorithm of electric power data network[J]. Journal of Beijing University of Posts and Telecommunications, 2017, 40(S1): 108-111.
[9] 段雪源, 付钰, 王坤. 基于VAE-WGAN的多维时间序列异常检测方法[J]. 通信学报, 2022, 43(3): 1-13.
DUAN X Y, FU Y, WANG K. Multi-dimensional time series anomaly detection method based on VAE-WGAN[J]. Journal on Communications, 2022, 43(3): 1-13.
[10] SHEN L, LI Z, KWOK J. Time series anomaly detection using temporal hierarchical one-class network[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS’20), 2020: 13016-13026.
[11] ZHAO H, WANG Y, DUAN J, et al. Multivariate time-series anomaly detection via graph attention network[C]// Proceedingds of the 2020 IEEE International Conference on Data Mining (ICDM 2020), Sorrento, 2020: 841-850.
[12] DENG A, HOOI B. Graph neural network-based anomaly detection in multivariate time series[C]//Proceedingds of the 35th AAAI Conference on Artificial Intelligence (AAAI 2021), 2021: 4027-4035.
[13] HUNDMAN K, CONSTANTINOU V, LAPORTE C, et al. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding[C]//Proceedingds of the 24th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (SIGKDD 2018), London, 2018: 387-395.
[14] CHEN Z, CHEN D, ZHANG X, et al. Learning graph structures with transformer for multivariate time series anomaly detection in IOT[J]. IEEE Internet of Things Journal, 2021, 9(12): 9179-9189.
[15] ZHANG C, SONG D, CHEN Y, et al. A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data[C]//Proceedingds of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019), Honolulu, 2019: 1409-1416.
[16] ZONG B, SONG Q, MIN M R, et al. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection[C]//Proceedingds of the 6th International Conference on Learning Representations (ICLR 2018), Vancouver, 2018.
[17] SU Y, ZHAO Y, NIU C, et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]// Proceedingds of the 25th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (SIGKDD 2019), Anchorag, 2019: 2828-2837.
[18] LI Z, ZHAO Y, HAN J, et al. Multivariate time series anomaly detection and interpretation using hierarchical inter-metric and temporal embedding[C]//Proceedingds of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (SIGKDD 2021), 2021: 3220-3230.
[19] ZHOU B, LIU S, HOOI B, et al. BeatGAN: anomalous rhythm detection using adversarially generated time series[C]//Proceedingds of the 28th International Joint Conference on Artificial Intelligence (IJCAI 2019), Macao, China, 2019: 4433-4439.
[20] AUDIBERT J, MICHIARDI P, GUYARD F, et al. USAD: unsupervised anomaly detection on multivariate time series[C]//Proceedingds of the 26th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (SIGKDD 2020). New York, N Y: ACM, 2020: 3395-3404.
[21] TULI S, CASALE G, JENNINGS N R. TranAD: deep transformer networks for anomaly detection in multivariate time series data[C]// Proceedingds of the 48th International Conference on Very Large Databases (VLDB 2022), Sydney, 2022.
[22] XU J, WU H, WANG J, et al. Anomaly transformer: time series anomaly detection with association discrepancy[C]//Proceedingds of the 10th International Conference on Learning Representations (ICLR 2022), 2022.
[23] KINGMA D P, WELLING M. Auto-encoding variational bayes[C]//Proceedingds of the 2nd International Conference on Learning Representations (ICLR 2014), Banff, 2014.
[24] DINH L, SOHL-DICKSTEIN J, BENGIO S. Density estimation using real nvp[C]//Proceedingds of the 5th International Conference on Learning Representations (ICLR 2017), Toulon, 2017.
[25] CHO K, VAN M B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[J]. arXiv:1406.1078, 2014.
[26] MATHUR A P, TIPPENHAUER N O. SWaT: a water treatment testbed for research and training on ICS security[C]// Proceedingds of the 2nd International Workshop on Cyber-Physical Systems for Smart Water Networks (CySWater 2016), Vienna, 2016: 31-36.
[27] AHMED C M, PALLETI V R, MATHUR A P. WADI: a water distribution testbed for research in the design of secure cyber physical systems[C]//Proceedingds of the 3rd International Workshop on Cyber?Physical Systems for Smart Water Networks (CySWater 2017), Pittsburgh, 2017: 25-28.
[28] GARG A, ZHANG W, SAMARAN J, et al. An evaluation of anomaly detection and diagnosis in multivariate time series[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 33(6): 2508-2517.