Anomaly Detection Method for Industrial IoT Timing Data

doi:10.3778/j.issn.1002-8331.2301-0197

Abstract

Abstract: As an important cornerstone of the fourth industrial revolution, the industrial Internet transforms isolated industrial systems into connected networks and is an important expansion direction for digital industrialization. Anomaly detection in the industrial Internet of things environment is of great significance for automated decision-making. To address the problem that most existing methods fail to effectively consider the complex unknown topological relationships between sensors and the multi-scale patterns inherent in the industrial IoT temporal data, an unsupervised anomaly detection method MSTSAD with multi-scale spatiotemporal feature fusion for industrial IoT temporal data is proposed, which firstly constructs a novel bi-directional spatiotemporal feature extraction module to sequentially captures the correlation and bidirectional dependency between multiple time series. Secondly, a multi-scale gated temporal convolutional neural network is designed to adaptively extract multi-scale temporal features of time series, and a dual affine projection is introduced to realize the cross-fusion of multi-scale temporal features and spatiotemporal features to enhance the feature extraction of the model on the original data. Finally, a variational self-encoder combined with adversarial training is proposed to amplify the reconstruction error of anomalies and enhance the anti-interference ability of the model to train data noise, which enhances the ability of model to discriminate anomalous data. Experiments are conducted on four publicly available datasets, GPW, ECG5000, Occupancy and SWaT, in comparison with five state-of-the-art methods, and the experimental results show that F1 scores of MSTSAD are enhanced by 0.015~0.047.

Key words: industrial Internet of things, anomaly detection, multi-scale, spatio-temporal information

摘要： 作为第四次工业革命重要基石，工业互联网将孤立的工业系统转化为连接的网络，是数字产业化的重要拓展方向，工业物联网环境中的异常检测对自动化决策具有重要意义。针对现有大多数方法未能有效地考虑传感器间复杂的未知拓扑关系以及工业物联网时序数据内在的不同尺度模式导致异常检测精度有待提升的问题，提出了一种面向工业物联网时序数据的多尺度时空特征融合无监督异常检测方法MSTSAD，构建了一种新颖的双向时空特征提取模块依次捕获多个时间序列之间的相关性和双向依赖性。设计了多尺度门控时间卷积神经网络自适应地提取时间序列的多尺度时序特征，并引入双仿射实现多尺度时序特征和时空特征的交叉融合，增强模型对原始数据的特征提取。提出了结合对抗训练的变分自编码器来放大异常的重构误差并增强模型对训练数据噪声的抗干扰能力，提高了模型对异常数据的区分能力。在GPW、ECG5000、Occupancy和SWaT四个公开数据集上与五种先进方法进行了对比实验，实验结果表明，MSTSAD的F1分数提升了0.015~0.047。

关键词: 工业物联网, 异常检测, 多尺度, 时空信息

XIE Wei, LU Shida, SHI Kuanzhi, WANG Honglan, GU Rongbin, HUANG Jun, LI Jing. Anomaly Detection Method for Industrial IoT Timing Data[J]. Computer Engineering and Applications, 2024, 60(12): 270-282.

谢伟, 卢士达, 时宽治, 王虹岚, 顾荣斌, 黄君, 李静. 面向工业物联网时序数据的异常检测方法[J]. 计算机工程与应用, 2024, 60(12): 270-282.

References

[1] RAUT R D, GOTMARE A, NARKHEDEB E, et al. Enabling technologies for industry 4.0 manufacturing and supply chain: concepts, current status, and adoption challenges[J]. IEEE Engineering Management Review, 2020, 48(2): 83-102.
[2] PATEL P, ALIM I, SHETH A. From raw data to smart manufacturing: AI and semantic Web of things for industry 4.0[J]. IEEE Intelligent Systems, 2018, 33(4): 79-86.
[3] LIANG W, HUANG W H, LONG J, et al. Deep reinforcement learning for resource protection and real-time detection in iot environment[J]. IEEE Internet of Things Journal, 2020, 7(7): 6392-6401.
[4] CAI Z P, HE Z B. Trading private range counting over big IoT data[C]//2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), 2019: 144-153.
[5] WU Y L, WANG Z H, MA Y X, et al. Deep reinforcement learning for blockchain in industrial IoT: a survey[J]. Computer Networks, 2021, 191(7): 108004.
[6] WU Y L, MA Y X, DAI H N, et al. Deep learning for privacy preservation in autonomous moving platforms enhanced 5G heterogeneous networks[J]. Computer Networks, 2021, 185(7): 107743.
[7] LIANG Y, CAI Z P, YU J G, et al. Deep learning based inference of private information using embedded sensors in smart devices[J]. IEEE Network, 2018, 32(4): 8-14.
[8] WANG H, MA S L, GUO C N, et al. Blockchain-based power energy trading management[J]. ACM Transactions on Internet Technology, 2021, 21(2): 1-16.
[9] CHANG C C, LIN C J. LIBSVM: a library for support vector machines[J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 1-27.
[10] ZHANG G P. Time series forecasting using a hybrid ARIMA and neural network model[J]. Neurocomputing, 2003, 50(1): 159-175.
[11] LIU J, SONG X, ZHOU Y, et al. Deep anomaly detection in packet payload[J]. Neurocomputing, 2022, 485: 205-218.
[12] ERGEN T, KOZAT S S. Unsupervised anomaly detection with LSTM neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(8): 3127-3141.
[13] ZHOU Y, SONG X, ZHANG Y, et al. Feature encoding with autoencoders for weakly supervised anomaly detection[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 33(6): 2454-2465.
[14] GOLDSTEIN M, DENGEL A. Histogram-based outlier score (hbos): a fast unsupervised anomaly detection algorithm[C]//35th German Conference on Artificial Intelligence, 2012: 59-63.
[15] RAMASWAMY S, RASTOGI R, SHIM K. Efficient algorithms for mining outliers from large data sets[C]//Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, 2000: 427-438.
[16] BREUNIG M M, KRIEGEL H P, NG R T, et al. LOF: identifying density-based local outliers[C]//Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, 2000: 93-104.
[17] AMER M, GOLDSTEIN M. Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer[C]//Proceedings of the 3rd RapidMiner Community Meeting and Conference (RCOMM 2012), 2012: 1-12.
[18] PAN X, TAN J, KAVULYA S, et al. Ganesha: BlackBox diagnosis of MapReduce systems[J]. ACM Sigmetrics Performance Evaluation Review, 2009, 37(3): 8-13.
[19] SCH?LKOPF B, PLATT J C, SHAWE-TAYLOR J, et al. Estimating the support of a high-dimensional distribution[J]. Neural Computation, 2001, 13(7): 1443-1471.
[20] TAX D M J, DUIN R P W. Support vector data description[J]. Machine Learning, 2004, 54(1): 45-66.
[21] MALHOTRA P, RAMAKRISHNAN A, ANAND G, et al. LSTM-based encoder-decoder for multi-sensor anomaly detection[J]. arXiv:1607.00148, 2016.
[22] KIM D, YANG H, CHUNG M, et al. Squeezed convolutional variational autoencoder for unsupervised anomaly detection in edge device industrial internet of things[C]//2018 International Conference on Information and Computer Technologies (ICICT), 2018: 67-71.
[23] AUDIBERT J, MICHIARDI P, GUYARD F, et al. USAD: unsupervised anomaly detection on multivariate time series[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020: 3395-3404.
[24] ZHANG C F, PENG K X, DONG J. An incipient fault detection and self-learning identification method based on robust SVDD and RBM-PNN[J]. Journal of Process Control, 2020, 85(1): 173-183.
[25] JOZEFOWICZ R, ZAREMBA W, SUTSKEVER I. An empirical exploration of recurrent network architectures[C]//2015 International Conference on Machine Learning (ICML), 2015: 2342-2350.
[26] GREFF K, SRIVASTAVA R K, KOUTNIK J, et al. LSTM: a search space odyssey[J]. IEEE Transactions on Neural Networks Learning Systems, 2017, 28(10): 2222-2232.
[27] BAI S J, KOLTER J Z, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[J]. arXiv:1803.01271, 2018.
[28] KROMANIS R, KRIPAKARAN P. Support vector regression for anomaly detection from measurement histories[J]. Advanced Engineering Informatics, 2013, 27(4): 486-495.
[29] HILL D, MINSKER B, AMIR E. Real-time Bayesian anomaly detection for environmental sensor data[C]//Proceedings of the Congress-International Association for Hydraulic Research, 2007: 503.
[30] SHIPMON D T, GUREVITCH J M, PISELLI P M, et al. Time series anomaly detection; detection of anomalous drops with limited features and sparse examples in noisy highly periodic data[J]. arXiv:1708.03665, 2017.
[31] REN H S, XU B X, WANG Y J, et al. Time-series anomaly detection service at microsoft[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019: 3009-3017.
[32] LI R F, CHEN H, FENG F X, et al. Dual graph convolutional networks for aspect-based sentiment analysis[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021: 6319-6329.
[33] NANDURI A, SHERRY L. Anomaly detection in aircraft data using recurrent neural networks (RNN)[C]//2016 Integrated Communications Navigation and Surveillance (ICNS), 2016.
[34] HUNDMAN K, CONSTANTINOU V, LAPORTE C, et al. Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2018: 387-395.
[35] XI Y H, TANG X, LI Z W, et al. Fault detection and classification on insulated overhead conductors based on MCNN-LSTM[J]. IET Renewable Power Generation, 2022, 16(7): 1425-1433.
[36] KUNDU S, ARI S. MsCNN: a deep learning framework for P300-based brain-computer interface speller[J]. IEEE Transactions on Medical Robotics and Bionics, 2019, 2(1): 86-93.
[37] DENG A L, HOOI B. Graph neural network-based anomaly detection in multivariate time series[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2021: 4027-4035.
[38] ZONG B, SONG Q, CHEN H F, et al. Deep autoencoding Gaussian mixture model for unsupervised anomaly detection[C]//International Conference on Learning Representations, 2018: 1-14.
[39] SU Y, ZHAO Y J, NIU C H, et al. Robust anomaly detection for multivariate time series through stochastic recurrent neural network[C]//Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019: 2828-2837.
[40] LIN S Y, CLARK R, BIRKE R, et al. Anomaly detection for time series using VAE-LSTM hybrid model[C]//2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020: 4322-4326.
[41] LIU F, ZHOU X S, CAO J L, et al. Anomaly detection in quasi-periodic time series based on automatic data segmentation and attentional LSTM-CNN[J]. IEEE Transactions on Knowledge and Data Engineering, 2022, 34(6): 2626-2640.
[42] LI D, CHEN D C, SHI L, et al. MAD-GAN: multivariate anomaly detection for time series data with generative adversarial networks[C]//International Conference on Artificial Neural Networks, 2019: 703-716.
[43] DAUPHIN Y N, FAN A, AULI M, et al. Language modeling with gated convolutional networks[C]//International Conference on Machine Learning, 2017: 933-941.
[44] SIFFER A, FOUQUE P A, TERMIER A, et al. Anomaly detection in streams with extreme value theory[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017: 1067-1075.
[45] CHEN Y P, HAO Y, RAKTHANMANON T. ECG5000: a general framework for never-ending learning from time series streams[J]. Data Mining Knowledge Discovery, 2015, 29(6): 1622-1664.
[46] LU K D, ZENG G Q, LUO X Z, et al. Evolutionary deep belief network for cyber-attack detection in industrial automation and control system[J]. IEEE Transactions on Industrial Informatics, 2021, 17(11): 7618-7627.
[47] CANDANEDO L M, FELDHEIM V. Occupancy: accurate occupancy detection of an office room from light, temperature, humidity and CO2 measurements using statistical learning models[J]. Engery and Buildings, 2016, 112(1): 28-39.
[48] MATHUR A P, TIPPENHAUER N O. SWaT: a water treatment testbed for research and training on ICS security[C]//2016 International Workshop on Cyber-Physical Systems for Smart Water Networks (CySWater), 2016: 31-36.
[49] XU H W, CHEN W X, ZHAO N W, et al. Unsupervised anomaly detection via variational auto-encoder for seasonal kpis in web applications[C]//Proceedings of the 2018 World Wide Web Conference, 2018: 187-196.
[50] ZHANG H, WANG Y J, DUAN J Y, et al. MTAD-GAT: multivariate time-series anomaly detection via graph attention network[C]//2020 IEEE Internation Conference on Data Mining (ICDM), 2020: 841-850.