互信息深度稀疏自编码融合DLSTM预测网络

doi:10.3778/j.issn.1002-8331.2103-0292

摘要/Abstract

摘要： 针对工业系统变量之间存在动态和相互关联特性导致关键变量预测精度降低问题，提出一种互信息深度堆叠稀疏自编码数据特征网络（mutual information-deep stack sparse auto-encoder，MI-DSSAE）结合深度长短期记忆（deep LSTM，DLSTM）预测模型。MI-DSSAE模型对稀疏编码器改进，采用堆叠稀疏编码器结构，引入互信息作为重构损失权重，对多个稀疏编码器隐层迁移并微调。预测部分采用深度DLSTM网络结构，用双层Bi-LSTM结构对序列数据的动态变化特性双向捕捉，将输出数据输入到普通LSTM进行记忆处理，进行全连接层加权预测关键质量变量。采用流程化工业案例脱丁烷塔的C4含量对提出的模型验证，同时对比RNN、LSTM、GRU模型以及MI-DSSAE-RNN、MI-DSSAE-LSTM、MI-DSSAE-GRU等模型，通过RMSE、R2和MAE多项回归误差指标对比分析，验证MI-DSSAE-DLSTM模型的有效性。

关键词: 深度稀疏自编码器, 互信息, 长短期记忆（LSTM）, 脱丁烷塔, 软测量

Abstract: In the view of dynamic and interrelated characteristics in industrial system variables, which leads to the problem of reduced prediction accuracy of key variables, a mutual information-deep stack sparse auto-encoder（MI-DSSAE） combined with deep long short-term memory（DLSTM） prediction model is proposed. MI-DSSAE improves the sparse encoder and superimposes the hidden layer into a stacked sparse encoder structure, introducing mutual information as the reconstruction loss weight, and then migrates and fine-tunes multiple sparse encoders. The prediction part adopts the deep two-way DLSTM network structure. Firstly, the two-layer Bi-LSTM structure is used to capture the dynamic characteristics of the sequence data in two directions, and the output data is input into the ordinary LSTM for memory processing. Finally, the fully connected layer is weighted to predict the key quality variables. The C4 content change of the debutanizer column in the process industry case is used to verify the proposed modeling, while RNN, LSTM, GRU, MI-DSSAE-RNN, MI-DSSAE-LSTM, and MI-DSSAE-GRU methods are compared with proposed method. The comprehensive comparative analysis of multiple regression error indicators of RMSE, R2, and MAPE shows the effectiveness of the MI-DSSAE-DLSTM modeling method.

Key words: deep sparse auto-encoder, mutual information, long short-term memory（LSTM）, debutanizer column, soft measurement

李江坤, 黄海燕. 互信息深度稀疏自编码融合DLSTM预测网络[J]. 计算机工程与应用, 2022, 58(20): 277-285.

LI Jiangkun, HUANG Haiyan. Mutual Information Deep Sparse Auto-Encoding Hybrid DLSTM Prediction Network[J]. Computer Engineering and Applications, 2022, 58(20): 277-285.

参考文献

[1] GAO X，YANG F，HUANG D，et al.An iterative two-level optimization method for the modeling of Wiener structure nonlinear dynamic soft sensors[J].Industrial & Engineering Chemistry Research，2014，53（3）：1172-1178.
[2] KHATIBISEPEHR S，HUANG B.Dealing with irregular data in soft sensors：Bayesian method and comparative study[J].Industrial & Engineering Chemistry Research，2008，47（22）：8713-8723.
[3] YUAN X，CHEN Z，WANG Y.Probabilistic nonlinear soft sensor modeling based on generative topographic mapping regression[J].IEEE Access，2018，6：10445-10452.
[4] CHOI S W，LEE I B.Nonlinear dynamic process monitoring based on dynamic kernel PCA[J].Chemical Engineering Science，2004，59（24）：5897-5908.
[5] WANG Y，SUN K，YUAN X，et al.A novel sliding window PCA-IPF based steady-state detection framework and its industrial application[J].IEEE Access，2018，5：20995-21004.
[6] CHEN N，DAI J，YUAN X，et al.Temperature prediction model for roller kiln by ALD-based double locally weighted kernel principal component regression[J].IEEE Transactions on Instrumentation and Measurement，2018，67（8）：2001-2010.
[7] NOMIKOS P，MACGREGOR J F.Multi-way partial least squares in monitoring batch processes[J].Chemometrics & Intelligent Laboratory Systems，1995，30（1）：97-108.
[8] KANEKO H，FUNATSU K.Adaptive soft sensor based on online support vector regression and Bayesian ensemble learning for various states in chemical plants[J].Chemometrics & Intelligent Laboratory Systems，2014，137：57-66.
[9] CHEN J，LIAO C M.Dynamic process fault monitoring based on neural network and PCA[J].Journal of Process Control，2002，12（2）：277-289.
[10] FORTUNA L，GIANNONE P，GRAZIANI S.Virtual instruments based on stacked neural networks to improve pro-
duct quality monitoring in a refinery[J].IEEE Transactions on Instrumentation and Measurement，2007，56（1）：95-101.
[11] SHANG C，YANG F，HUANG D，et al.Data-driven soft sensor development based on deep learning technique[J].Journal of Process Control，2014，24（3）：223-233.
[12] YAO L，GE Z.Deep learning of semi-supervised process data with hierarchical extreme learning machine and soft sensor application[J].IEEE Transactions on Industrial Electronics，2017，65（2）：1490-1498.
[13] YUAN X，WANG Y，YANG C，et al.Weighted linear dynamic system for feature representation and soft sensor application in nonlinear dynamic industrial processes[J].IEEE Transactions on Industrial Electronics，2018，65（2）：1508-1517.
[14] KASPAR M H，RAY W H.Chemometric methods for process monitoring and high performance controller design[J].AIChE Journal，1992，38（10）：1593-1608.
[15] GRAVES A.Supervised sequence labelling with recurrent neural networks[M]//Studies in computational intelligence.[S.l.]：Springer，2012.
[16] LI D，LI Z，SUN K.Development of a novel soft sensor with long short-term memory network and normalized mutual information feature selection[J].Mathematical Problems in Engineering，2020（2）：1-11.
[17] 王硕，王培良.基于深层长短期记忆网络与批规范化的间歇过程故障检测方法[J].计算机应用，2019，39（2）：370-375.
WANG S，WANG P L.Fault detection method for batch process based on deep long short-term memory network and batch normalization[J].Journal of Computer Applications，2019，39（2）：370-375.
[18] 刘炳春，来明昭，齐鑫，等.基于Wavelet-LSTM模型的北京空气污染物浓度预测[J].环境科学与技术，2019（8）：142-149.
LIU B C，LAI M Z，QI X，et al.Forecasting the air pollutant concentration in Beijing based on Wavelet-LSTM model[J].Environmental Science & Technology，2019（8）：142-149.
[19] YUAN X，LI L，WANG Y.Nonlinear dynamic soft sensor modeling with supervised long short-term memory network[J].IEEE Transactions on Industrial Informatics，2020：3168-3176.
[20] SHAO W，TIAN X.Semi-supervised selective ensemble learning based on distance to model for nonlinear soft sensor development[J].Neurocomputing，2016，222：91-104.
[21] YUAN X，HUANG B，WANG Y，et al.Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted SAE[J].IEEE Transactions on Industrial Informatics，2018，14（7）：3235-3243.
[22] 袁延鑫，孙莉，张群.基于堆栈稀疏自编码器和微动特征的身份认证技术[J].空军工程大学学报（自然科学版），2018，19（4）：48-53.
YUAN Y X，SUN L，ZHANG Q.Authentication techno-
logy via stack sparse autoencoder and micro motion feature[J].Journal of Air Force Engineering University（Natural Science Edition），2018，19（4）：48-53.
[23] 罗金，童靳于，郑近德，等.基于EEMD和堆叠稀疏自编码的滚动轴承故障诊断方法[J].噪声与振动控制，2020，40（2）：120-125.
LUO J，TONG J Y，ZHENG J D，et al.Fault diagnosis method for rolling bearings based on EEMD and stacked sparse auto-encoder[J].Noise and Vibration Control，2020，40（2）：120-125.
[24] SL A，DSB C，AGD E，et al.Stable feature selection using copula based mutual information[J].Pattern Recognition，2020，112（1）：107697.
[25] 滕金保，孔韦韦，田乔鑫，等.基于LSTM-Attention与CNN混合模型的文本分类方法[J].计算机工程与应用，2021，57（14）：126-133.
TENG J B，KONG W W，TIAN Q X，et al.Text classification method based on LSTM-attention and CNN hybrid model[J].Computer Engineering and Applications，2021，57（14）：126-133.
[26] 唐永旺，刘欣.基于Bi-LSTM和自注意力的恶意代码检测方法[J].计算机应用与软件，2021，38（3）：327-333.
TANG Y W，LIU X.A malicious code detection method based on Bi-LSTM and self-attention[J].Computer Applications and Software，2021，38（3）：327-333.
[27] PANI A K，AMIN K G，MOHANTA H K.Soft sensing of product quality in the debutanizer column with principal component analysis and feed-forward artificial neural network[J].Alexandria Engineering Journal，2016，55（2）：1667-1674.