Research Progress of Deep Learning in Fault Diagnosis of Chemical Process Industry

doi:10.3778/j.issn.1002-8331.2111-0368

Abstract

Abstract: Chemical process industry fault diagnosis（CPIFD） is an important branch of intelligent manufacturing. In recent years, deep learning has demonstrated unique advantages and potential in feature recognition and classification. Therefore, the research of CPIFD based on deep learning has received extensive attention from scholars. However, there are limited reviews about CPIFD based on deep learning in the published literature. Hence, the purpose of the paper is to offer the state-of-the-art reference for CPIFD and stimulate scholars to further explore the application of deep learning in CPIFD. Firstly, the development of CPIFD technology is described. Secondly, the fundamental theories of representative models in deep learning are expounded and their applications in CPIFD are reviewed. These models include convolutional neural network, deep belief network, stacked autoencoder, long short-term memory network and other emerging neural network models. Finally, the problems faced by deep learning in CPIFD are discussed and the outlooks on future directions worthy of research are also presented.

Key words: process industry, fault diagnosis, deep learning, feature extraction, chemical process

摘要： 化学流程工业故障诊断（chemical process industry fault diagnosis，CPIFD）是智能制造的一个重要分支。近年来，深度学习在特征识别和分类方面显示出独特的优势和潜力，因此，基于深度学习的CPIFD研究受到了学者们的广泛关注。然而，在已发表的研究文献中，关于基于深度学习的CPIFD的论述是有限的，因此，旨在为CPIFD的研究提供最新的参考，并激励学者进一步探讨深度学习在CPIFD中的应用。介绍了CPIFD技术的发展，阐述了在深度学习中具有代表性模型的基本理论，并综述了它们在CPIFD中的应用，这些模型包括卷积神经网络、深度置信网络、堆叠自动编码器、长短期记忆网络和其他新兴神经网络模型；讨论了深度学习在CPIFD中所面临的问题，并对今后值得研究的方向提出了展望。

关键词: 流程工业, 故障诊断, 深度学习, 特征提取, 化工过程

CHEN Honghua, CEN Jian, LIU Xi, YANG Zhuohong. Research Progress of Deep Learning in Fault Diagnosis of Chemical Process Industry[J]. Computer Engineering and Applications, 2022, 58(13): 48-62.

陈红花, 岑健, 刘溪, 杨卓洪. 深度学习在化学流程工业故障诊断的研究进展[J]. 计算机工程与应用, 2022, 58(13): 48-62.

References

[1] 车建国，赵赛.基于数据深度的过程工业故障检测方法[J].计算机工程与应用，2020，56（1）：265-271.
CHE J G，ZHAO S.Fault detection method based on data depth for process industry[J].Computer Engineering and Applications，2020，56（1）：265-271.
[2] ARUNTHAVANATHAN R，KHAN F，AHMED S，et al.An analysis of process fault diagnosis methods from safety perspectives[J].Computers & Chemical Engineering，2021，145：107197.
[3] JACKSON J E.A user’s guide to principal components[M].New York：John Wiley & Sons，2005.
[4] WOLD S，KETTANEH-WOLD N，SKAGERBERG B.Nonlinear PLS modeling[J].Chemometrics and Intelligent Laboratory Systems，1989，7（1）：53-65.
[5] HYV?RINEN A，OJA E.Independent component analysis：algorithms and applications[J].Neural Networks，2000，13（4/5）：411-430.
[6] 孔祥玉，杨治艳，刘佑民，等.基于独立成分分析及其扩展模型的工业过程监测方法综述[J].控制与决策，2022，37（4）：799-814.
KONG X Y，YANG Z Y，LIU Y M，et al.Overview of industrial process monitoring methods based on independent component analysis and its extended model[J].Control and Decision，2022，37（4）：799-814.
[7] HU Q，QIN A，ZHANG Q，et al.Fault diagnosis based on weighted extreme learning machine with wavelet packet decomposition and KPCA[J].IEEE Sensors Journal，2018，18（20）：8472-8483.
[8] FENG L，ZHANG Y，LI X，et al.Independent component analysis based on data-driven reconstruction of multi-fault diagnosis[J].Journal of chemometrics，2017，31（12）：e2923.
[9] ZHANG Y，ZHOU H，QIN S J，et al.Decentralized fault diagnosis of large-scale processes using multi-block kernel partial least squares[J].IEEE Transactions on Industrial Informatics，2010，6（1）：3-10.
[10] EL-SHAL S M，MORRIS A S.A fuzzy expert system for fault detection in statistical process control of industrial processes[J].IEEE Transactions on Systems，Man and Cybernetics，Part C（Applications and Reviews），2000，30（2）：281-289.
[11] ZHANG Y，MA C.Fault diagnosis of nonlinear processes using multiscale KPCA and multiscale KPLS[J].Chemical Engineering Science，2011，66（1）：64-72.
[12] AJAMI A，DANESHVAR M.Data driven approach for fault detection and diagnosis of turbine in thermal power plant using independent component analysis（ICA）[J].International Journal of Electrical Power & Energy Systems，2012，43（1）：728-735.
[13] STEFATOS G，BEN HAMZA A.Dynamic independent component analysis approach for fault detection and diagnosis[J].Expert Systems with Applications，2010，37（12）：8606-8617.
[14] YIN S，DING S X，XIE X，et al.A review on basic data-driven approaches for industrial process monitoring[J].IEEE Transactions on Industrial Electronics，2014，61（11）：6418-6428.
[15] YIN Z，HOU J.Recent advances on SVM based fault diagnosis and process monitoring in complicated industrial processes[J].Neurocomputing（Amsterdam），2016，174：643-650.
[16] REN L，LV W，JIANG S，et al.Fault diagnosis using a joint model based on sparse representation and SVM[J].IEEE Transactions on Instrumentation and Measurement，2016，65（10）：2313-2320.
[17] SONG B，TAN S，SHI H，et al.Fault detection and diagnosis via standardized k nearest neighbor for multimode process[J].Journal of the Taiwan Institute of Chemical Engineers，2020，106：1-8.
[18] XIONG J，ZHANG Q，SUN G，et al.An information fusion fault diagnosis method based on dimensionless indicators with static discounting factor and KNN[J].IEEE Sensors Journal，2016，16（7）：2060-2069.
[19] SHI Q，ZHANG H.Fault diagnosis of an autonomous vehicle with an improved SVM algorithm subject to unbalanced datasets[J].IEEE Transactions on Industrial Electronics，2021，68（7）：6248-6256.
[20] CAO S，HU Z，LUO X，et al.Research on fault diagnosis technology of centrifugal pump blade crack based on PCA and GMM[J].Measurement，2021，173：108558.
[21] HAJNAYEB A，GHASEMLOONIA A，KHADEM S E，et al.Application and comparison of an ANN-based feature selection method and the genetic algorithm in gearbox fault diagnosis[J].Expert Systems with Applications，2011，38（8）：10205-10209.
[22] HE S，LIU X，WANG Y，et al.An effective fault diagnosis approach based on optimal weighted least squares support vector machine[J].The Canadian Journal of Chemical Engineering，2017，95（12）：2357-2366.
[23] 于春梅.稀疏特征选择在过程工业故障诊断中的应用[J].计算机工程与应用，2014，50（18）：257-260.
YU C M.Sparse feature selection method for fault diagnosis of process industry[J].Computer Engineering and Applications，2014，50（18）：257-260.
[24] HINTON G E，SALAKHUTDINOV R R.Reducing the dimensionality of data with neural networks[J].Science（American Association for the Advancement of Science），2006，313：504-507.
[25] TAMILSELVAN P，WANG P.Failure diagnosis using deep belief learning based health state classification[J].Reliability Engineering & System Safety，2013，115：124-135.
[26] XIE D，BAI L.A hierarchical deep neural network for fault diagnosis on Tennessee-Eastman process[C]//Proceedings of the IEEE 14th International Conference on Machine Learning and Applications（ICMLA），2015：745-748.
[27] ISERMANN R.Model-based fault-detection and diagnosis-status and applications[J].Annual Reviews in Control，2005，29（1）：71-85.
[28] WANG H，CHAI T，DING J，et al.Data driven fault diagnosis and fault tolerant control：Some advances and possible new directions[J].Acta Automatica Sinica，2009，35（6）：739-747.
[29] DAI X，GAO Z.From model，signal to knowledge：A data-driven perspective of fault detection and diagnosis[J].IEEE Transactions on Industrial Informatics，2013，9（4）：2226-2238.
[30] XU Y，SUN Y，WAN J，et al.Industrial big data for fault diagnosis：Taxonomy，review，and applications[J].IEEE Access，2017，5：17368-17380.
[31] NOR N M，HASSAN C R C，HUSSAIN M A.A review of data-driven fault detection and diagnosis methods：Applications in chemical process systems[J].Reviews in Chemical Engineering，2019，36（4）：513-553.
[32] JIAO Z，HU P，XU H，et al.Machine learning and deep learning in chemical health and safety：A systematic review of techniques and applications[J].Journal of Chemical Health & Safety（Online），2020，27（6）：316-334.
[33] TAQVI S A A，ZABIRI H，TUFA L D，et al.A review on data-driven learning approaches for fault detection and diagnosis in chemical processes[J].Chem Bio Eng Reviews，2021，8（3）：239-259.
[34] KHAN A，SOHAIL A，ZAHOORA U，et al.A survey of the recent architectures of deep convolutional neural networks[J].The Artificial Intelligence Review，2020，53（8）：5455-5516.
[35] RAWAT W，WANG Z.Deep convolutional neural networks for image classification：A comprehensive review[J].Neural Computation，2017，29（9）：2352-2449.
[36] YAO Y，WANG H，LI S，et al.End-to-end convolutional neural network model for gear fault diagnosis based on sound signals[J].Applied Sciences，2018，8（9）：1584.
[37] CHEN Z，CEN J，XIONG J.Rolling bearing fault diagnosis using time-frequency analysis and deep transfer convolutional neural network[J].IEEE Access，2020，8：150248-150261.
[38] WU H，ZHAO J.Deep convolutional neural network model based chemical process fault diagnosis[J].Computers & Chemical Engineering，2018，115：185-197.
[39] ZHANG H，WANG P，GAO X，et al.Amplitude-frequency images-based ConvNet：Applications of fault detection and diagnosis in chemical processes[J].Journal of Chemometrics，2019，33（9）：e3168.
[40] GAO X，YANG F，FENG E.A process fault diagnosis method using multi-time scale dynamic feature extraction based on convolutional neural network[J].Canadian Journal of Chemical Engineering，2020，98（6）：1280-1292.
[41] YU J，ZHANG C，WANG S.Multichannel one-dimensional convolutional neural network-based feature learning for fault diagnosis of industrial processes[J].Neural Computing & Applications，2021，33（8）：3085-3104.
[42] YU W，ZHAO C.Broad convolutional neural network based industrial process fault diagnosis with incremental learning capability[J].IEEE Transactions on Industrial Electronics，2020，67（6）：5081-5091.
[43] AZAMFAR M，LI X，LEE J.Deep learning-based domain adaptation method for fault diagnosis in semiconductor manufacturing[J].IEEE Transactions on Semiconductor Manufacturing，2020，33（3）：445-453.
[44] PAN T，CHEN J，ZHOU Z.Intelligent fault diagnosis of rolling bearing via deep-layer wise feature extraction using deep belief network[C]//Proceedings of the International Conference on Sensing，Diagnostics，Prognostics，and Control（SDPC），2018：509-514.
[45] LIU J，LIU Y，LUO X.Research and development on Boltzmann machine[J].Journal of Computer Research and Development，2014，51（1）：1.
[46] MOVAHEDI F，COYLE J L，SEJDIC E.Deep belief networks for electroencephalography：A review of recent contributions and future outlooks[J].IEEE Journal of Biomedical and Health Informatics，2018，22（3）：642-652.
[47] ERHAN D，COURVILLE A，BENGIO Y，et al.Why does unsupervised pre-training help deep learning?[C]//Proceedings of the 13th International Conference on Artificial Intelligence and Statistics，2010：201-208.
[48] SCHMIDHUBER J.Deep learning in neural networks：An overview[J].Neural Networks，2015，61：85-117.
[49] YU J，LIU G.Knowledge extraction and insertion to deep belief network for gearbox fault diagnosis[J].Knowledge-Based Systems，2020，197：105883.
[50] ZHANG Z，ZHAO J.A deep belief network based fault diagnosis model for complex chemical processes[J].Computers & Chemical Engineering，2017，107：395-407.
[51] 张祥，崔哲，董玉玺，等.基于VAE_DBN的故障分类方法在化工过程中的应用[J].过程工程学报，2018，18（3）：590-594.
ZHANG X，CUI Z，DONG Y X，et al.Application of fault classification method based on VAE-DBN in chemical process[J].The Chinese Journal of Process Engineering，2018，18（3）：590-594.
[52] TANG Q，CHAI Y，QU J，et al.Fisher discriminative sparse representation based on DBN for fault diagnosis of complex system[J].Applied Sciences，2018，8（5）：795.
[53] WANG Y，PAN Z，YUAN X，et al.A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network[J].ISA Transactions，2020，96：457-467.
[54] WEI Y，WENG Z.Research on TE process fault diagnosis method based on DBN and dropout[J].The Canadian Journal of Chemical Engineering，2020，98（6）：1293-1306.
[55] AJAGEKAR A，YOU F.Quantum computing assisted deep learning for fault detection and diagnosis in industrial process systems[J].Computers & Chemical Engineering，2020，143：107119.
[56] RUMELHART D E，HINTON G E，WILLIAMS R J.Learning representations by back-propagating errors[J].Nature（London），1986，323：533-536.
[57] RUMELHART D E，MCCLELLAND J L.Learning internal representations by error propagation[R].California Univ San Diego La Jolla Inst for Cognitive Science，1985.
[58] VINCENT P，LAROCHELLE H，LAJOIE I，et al.Stacked denoising autoencoders：learning useful representations in a deep network with a local denoising criterion[J].Journal of Machine Learning Research，2010，11（12）：3371-3408.
[59] VINCENT P，LAROCHELLE H，BENGIO Y，et al.Extracting and composing robust features with denoising autoencoders[C]//Proceedings of the 25th International Conference on Machine Learning，2008：1096-1103.
[60] NG A.Sparse autoencoder[J].CS294A Lecture Notes，2011，72：1-19.
[61] MASCI J，MEIER U，CIRE?AN D，et al.Stacked convolutional auto-encoders for hierarchical feature extraction[C]//Proceedings of the International Conference on Artificial Neural Networks，2011：52-59.
[62] LV F，WEN C，BAO Z，et al.Fault diagnosis based on deep learning[C]//Proceedings of the American Control Conference（ACC），2016：6851-6856.
[63] LV F，WEN C，LIU M，et al.Weighted time series fault diagnosis based on a stacked sparse autoencoder[J].Journal of Chemometrics，2017，31（9）：e2912.
[64] ZHENG S，ZHAO J.A new unsupervised data mining method based on the stacked autoencoder for chemical process fault diagnosis[J].Computers & Chemical Engineering，2020，135：106755.
[65] CHEN S，YU J，WANG S.One-dimensional convolutional auto-encoder-based feature learning for fault diagnosis of multivariate processes[J].Journal of Process Control，2020，87：54-67.
[66] LI Z，TIAN L，JIANG Q，et al.Fault diagnostic method based on deep learning and multi-model feature fusion for complex industrial processes[J].Industrial & Engineering Chemistry Research，2020，59：18061-18069.
[67] ZHANG C，YU J，WANG S.Fault detection and recognition of multivariate process based on feature learning of one-dimensional convolutional neural network and stacked denoised autoencoder[J].International Journal of Production Research，2021，59（8）：2426-2449.
[68] CHADHA G S，PANAMBILLY A，SCHWUNG A，et al.Bidirectional deep recurrent neural networks for process fault classification[J].ISA Transactions，2020，106：330-342.
[69] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[70] YONG Y，XIAOSHENG S，CHANGHUA H，et al.A review of recurrent neural networks：LSTM cells and network architectures[J].Neural Computation，2019，31（7）：1235-1270.
[71] ZHAO H，SUN S，JIN B.Sequential fault diagnosis based on lstm neural network[J].IEEE Access，2018，6：12929-12939.
[72] PARK P，MARCO P D，SHIN H，et al.Fault detection and diagnosis using combined autoencoder and long short-term memory network[J].Sensors（Basel，Switzerland），2019，19（21）：4612.
[73] SHAO B，HU X，BIAN G，et al.A multichannel LSTM-CNN method for fault diagnosis of chemical process[J].Mathematical Problems in Engineering，2019（3）：1-14.
[74] WANG N，YANG F，ZHANG R，et al.Intelligent fault diagnosis for chemical processes using deep learning multi-model fusion[J].IEEE Transactions on Cybernetics，2020：1-15.
[75] YUAN J，TIAN Y.A multiscale feature learning scheme based on deep learning for industrial process monitoring and fault diagnosis[J].IEEE Access，2019，7：151189-151202.
[76] PAN S J，YANG Q.A Survey on transfer learning[J].IEEE Transactions on Knowledge and Data Engineering，2009，22（10）：1345-1359.
[77] WU H，ZHAO J.Fault detection and diagnosis based on transfer learning for multimode chemical processes[J].Computers & Chemical Engineering，2020，135：106731.
[78] WANG Y，WU D，YUAN X.LDA-based deep transfer learning for fault diagnosis in industrial chemical processes[J].Computers & Chemical Engineering，2020，140：106964.
[79] WANG J，ZHANG W，WU H，et al.Improved bilayer convolution transfer learning neural network for industrial fault detection[J/OL].Canadian Journal of Chemical Engineering（2021-07-31）.https：//doi.org/10.1002/cjce.24281.
[80] XIAO Y，SHI H，WANG B，et al.Adaptive manifold discriminative distribution alignment for fault diagnosis of chemical processes[J].Industrial & Engineering Chemistry Research，2021，60：9860-9870.
[81] LI W，GU S，ZHANG X，et al.Transfer learning for process fault diagnosis：Knowledge transfer from simulation to physical processes[J].Computers & Chemical Engineering，2020，139：106904.
[82] 朱张莉，饶元，吴渊，等.注意力机制在深度学习中的研究进展[J].中文信息学报，2019，33（6）：1-11.
ZHU Z L，RAO Y，WU Y，et al.Research progress of attention mechanism in deep learning[J].Journal of Chinese Information Processing，2019，33（6）：1-11.
[83] ZHANG K，TANG B，DENG L，et al.A hybrid attention improved ResNet based fault diagnosis method of wind turbines gearbox[J].Measurement：Journal of the International Measurement Confederation，2021，179：109491.
[84] 张智禹，尹爱军，谭建.融合注意力机制的改进DBN变工况齿轮箱故障诊断方法[J].振动与冲击，2021，40（14）：47-52.
ZHANG Z Y，YIN A J，TAN J.Improved DBN method with attention mechanism for the fault diagnosis of gearboxes under varying working condition[J].Journal of Vibration and Shock，2021，40（14）：47-52.
[85] MU K，LUO L，WANG Q，et al.Industrial process monitoring and fault diagnosis based on temporal attention augmented deep network[J].Journal of Information Processing Systems，2021，17（2）：242-252.
[86] LI S，LUO J，HU Y.Nonlinear process modeling via unidimensional convolutional neural networks with self-attention on global and local inter-variable structures and its application to process monitoring[J].ISA Transactions，2022，121：105-118.
[87] BI X，ZHAO J.A novel orthogonal self-attentive variational autoencoder method for interpretable chemical process fault detection and identification[J].Process Safety and Environmental Protection，2021，156：581-597.
[88] 刘兴，余建波.注意力卷积GRU自编码器及其在工业过程监控的应用[J].浙江大学学报（工学版），2021，55（9）：1643-1651.
LIU X，YU J B.Attention convolutional GRU-based autoencoder and its application in industrial process monitoring[J].Journal of Zhejiang University（Engineering Science），2021，55（9）：1643-1651.
[89] LI T，ZHOU Z，LI S，et al.The emerging graph neural networks for intelligent fault diagnostics and prognostics：A guideline and a benchmark study[J].Mechanical Systems and Signal Processing，2022，168：108653.
[90] 徐冰冰，岑科廷，黄俊杰，等.图卷积神经网络综述[J].计算机学报，2020，43（5）：755-780.
XU B B，CEN K T，HUANG J J，et al.A survey on graph convolutional neural network[J]，Chinese Journal of Computers，2020，43（5）：755-780.
[91] WU D，ZHAO J.Process topology convolutional network model for chemical process fault diagnosis[J].Process Safety and Environmental Protection，2021，150：93-109.
[92] WENDE T，ZIJIAN L，LENING L，et al.Identification of abnormal conditions in high-dimensional chemical process based on feature selection and deep learning[J].Chinese Journal of Chemical Engineering，2020，28（7）：1875-1883.