Representation and Classification of Echo State Network Models for Multivariate Time Series

doi:10.3778/j.issn.1002-8331.2204-0344

Abstract

Abstract: The reservoir of echo state networks（ESN） can not only fully mine the dynamics of time series but also further improve the training efficiency. However, the current ESN-based algorithms are difficult to achieve the accuracy of the complex neural network. A multivariate time series learning and classification method based on generative model distance is proposed. Firstly, the low-dimensional dynamic raw input is mapped into a high-dimensional static state space, thanks to the advantages of ESN in dynamic data representation. Then the generative model of the reservoir states is learned as the model representation. Finally, the classification is based on the distance between the prototype and the input in the space formed by the function model setwith prototype reasoning. The results can be derived and explained by its similar prototype in the model readout space. Experiments on benchmark datasets verify the advantages of the method in real-time performance and classification performance.

Key words: multivariate time series classification, echo state network, model space, prototype learning

摘要： 回声状态网络（echo state network，ESN）的储备池结构不仅能充分挖掘序列数据中动态信息，也进一步提高了训练效率。然而目前基于ESN的算法难以达到复杂神经网络的精度，为此提出一种基于生成模型距离度量的多元时间序列学习与分类方法。利用ESN在动态数据表示的优势将低维动态原始输入映射到高维静态空间，再拟合储备池状态序列的生成模型作为数据的模型表达，结合原型推理，基于生成模型集合张成的空间中原型与输入的距离进行分类，其结果能通过在模型读出空间的相似原型来推导，具有可解释性。基准数据集上的实验验证了该方法在算法实时性和分类性能上的优势。

关键词: 多元时间序列, 回声状态网络, 模型空间, 原型学习

HE Sha, ZHOU Xiren, CHEN Qiuju. Representation and Classification of Echo State Network Models for Multivariate Time Series[J]. Computer Engineering and Applications, 2023, 59(15): 132-140.

何莎, 周熙人, 陈秋菊. 多元时间序列的回声状态网络模型表达与分类[J]. 计算机工程与应用, 2023, 59(15): 132-140.

References

[1] ISMAIL FAWAZ H，FORESTIER G，WEBER J，et al.Deep learning for time series classification：a review[J].Data Mining and Knowledge Discovery，2019，33（4）：
[2] HUANG Z，YANG C，CHEN X，et al.Functional deep echo state network improved by a bi-level optimization approach for multivariate time series classification[J].Applied Soft Computing，2021，106：107314.
[3] TANAKA G，YAMANE T，HéROUX J B，et al.Recent advances in physical reservoir computing：a review[J].Neural Networks，2019，115：100-123.
[4] EVANUSA M，SHRESTHA S，PATIL V，et al.Deep-readout random recurrent neural networks for real-world temporal data[J].SN Computer Science，2022，3（3）：1-12.
[5] BERNDT D J，CLIFFORD J.Using dynamic time warping to find patterns in time series[C]//Proceedings of AAAI-94 Workshop on Knowledge Discovery in Databases，1994：229-248.
[6] ABANDA A，MORI U，LOZANO J A.A review on distance based time series classification[J].Data Mining and Knowledge Discovery，2019，33（2）：378-412.
[7] YAHIA H S，ABDULAZEEZ A M.Medical text classification based on convolutional neural network：a review[J].International Journal of Science and Business，2021，5（3）：27-41.
[8] CHEN H，TI?O P，RODAN A，et al.Learning in the model space for cognitive fault diagnosis[J].IEEE Transactions on Neural Networks and Learning Systems，2013，25（1）：124-136.
[9] ASWOLINSKIY W，REINHART R F，STEIL J.Time series classification in reservoir-and model-space[J].Neural Processing Letters，2018，48（2）：789-809.
[10] LI O，LIU H，CHEN C，et al.Deep learning for case-based reasoning through prototypes：a neural network that explains its predictions[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[11] GEE A H，GARCIA-OLANO D，GHOSH J，et al.Explaining deep classification of time-series data with learned prototypes[C]//CEUR Workshop Proceedings，2019：15.
[12] RODAN A，TI?O P.Simple deterministically constructed cycle reservoirs with regular jumps[J].Neural Computation，2012，24（7）：1822-1852.
[13] BIANCHI F M，SCARDAPANE S，L?KSE S，et al.Reservoir computing approaches for representation and classification of multivariate time series[J].IEEE Transactions on Neural Networks and Learning Systems，2020，32（5）：2169-2179
[14] GONG Z，CHEN H.Model-based oversampling for imbalanced sequence classification[C]//Proceedings of the 25th ACM International on Conference on Information and Knowledge Management，2016：1009-1018.
[15] CHEN H，TANG F，TINO P，et al.Model-based kernel for efficient time series analysis[C]//Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2013：392-400.
[16] CHEN Y，KEOGH E，HU B，et al.The ucr time series classification archive[EB/OL].（2015）.http：//www.cs.ucr.edu/~eamonn/time_series_data/.
[17] DUA D，GRAFF C.UCI machine learning repository[D/OL].University of California，Irvine，School of Information and Computer Sciences，2017.http：//archive.ics.uci.edu/ml.
[18] HOCHREITER S，SCHMIDHUBER J.Long short-term memory[J].Neural Computation，1997，9（8）：1735-1780.
[19] VAN DER MAATEN L.Learning discriminative fisher kernels[C]//Proceedings of the 28th International Conference on Machine Learning，2011：217-224.