增强局部注意力的时间序列分类方法

doi:10.3778/j.issn.1002-8331.2207-0444

摘要/Abstract

摘要： 现有时间序列分类方法普遍基于一种循环网络结构解决时间序列点值耦合问题，无法并行计算，导致计算资源浪费，因此提出一种增强局部注意力的时间序列分类方法。该方法拟合混合距离信息以增加时间序列位置感知能力，将混合距离信息融入自注意矩阵计算中，从而扩展自注意力机制；构造多尺度卷积注意力获取多尺度局部前向信息，以解决标准自注意力机制基于点值计算存在注意力混淆的问题；使用改进后的自注意力机制构造时序自注意分类模块，并行计算处理时间序列分类任务。实验结果表明，与现有时间序列分类方法相比，基于局部注意力增强的时间序列分类方法能够加速收敛，有效提高时序序列分类效果。

关键词: 时间序列分类, 自注意力机制, 位置感知, 多尺度卷积

Abstract: Existing time series classification methods are generally based on a circular network structure to solve the point value coupling problem of time series, which cannot be computed in parallel, resulting in a waste of computing resources. Therefore, this paper proposes a time series classification method with local attention enhancement. The mixed distance information is fitted to increase the position information perception of time series, the mixed distance information is incorporated into the self-attention matrix calculation to expand the self-attention mechanism. Multi-scale convolution attention is constructed to obtain multi-scale local forward information to solve the attention confusion problem in point value calculation of standard self-attention mechanism. The improved self-attention mechanism is used to construct the sequential self-attention classification module, and the time series classification task is processed by parallel computation. The experimental results show that, compared with the existing time series classification methods, the time series classification method based on local attention enhancement can accelerate convergence and effectively improve the classification effect of time series.

Key words: time series classification, self-attention mechanism, position perception, multi-scale convolution

李克文, 柯翠虹, 张敏, 王晓晖, 耿文亮. 增强局部注意力的时间序列分类方法[J]. 计算机工程与应用, 2024, 60(1): 189-197.

LI Kewen, KE Cuihong, ZHANG Min, WANG Xiaohui, GENG Wenliang. Time Series Classification Method with Local Attention Enhancement[J]. Computer Engineering and Applications, 2024, 60(1): 189-197.

参考文献

[1] 杨梦晨, 陈旭栋, 蔡鹏, 等. 早期时间序列分类方法研究综述[J]. 华东师范大学学报 (自然科学版), 2021(5): 115-133. YANG M C, CHEN X D, CAI P, et al. An overview of early time series classification methods[J]. Journal of East China Normal University (Natural Science), 2021(5): 115-133.
[2] 张雅雯, 王志海, 刘海洋, 等. 基于多尺度残差FCN的时间序列分类算法[J]. 软件学报, 2022, 33(2): 555-570.
ZHANG Y W, WANG Z H, LIU H Y, et al. Time series classification algorithm based on multi-scale residual FCN[J]. Journal of Software, 202, 33(2): 555-570.
[3] 李海林, 贾瑞颖, 谭观音. 基于K-Shape的时间序列模糊分类方法[J]. 电子科技大学学报, 2021, 50(6): 899-906.
LI H L, JIA R Y, TAN G Y. Fuzzy classification method of time series based on K-Shape[J]. Journal of University of Electronic Science and Technology of China, 2021, 50(6): 899-906.
[4] XIAO Z, XU X, XING H, et al. RTFN: a robust temporal feature network for time series classification[J]. Information Sciences, 2021, 571: 65-86.
[5] KWON D H, KIM J B, HEO J S, et al. Time series classification of cryptocurrency price trend based on a recurrent LSTM neural network[J]. Journal of Information Processing Systems, 2019, 15(3): 694-706.
[6] ELSAYED N, MAIDA A S, BAYOUMI M. Deep gated recurrent and convolutional network hybrid model for univariate time series classification[J]. International Journal of Advanced Computer Science and Applications, 2019, 10(5).
[7] 李向伟, 刘思言, 高昆仑. 基于双向长短时记忆网络和卷积神经网络的电力系统暂态稳定评估[J]. 科学技术与工程, 2020, 20(7): 2733-2739.
LI X W, LIU S Y, GAO K L. Transient stability assessment of power system based on bidirectional long and short memory network and convolutional neural network[J]. Science Technology and Engineering, 2020, 20(7): 2733-2739.
[8] LI T, ZHANG Y, WANG T. SRPM-CNN: a combined model based on slide relative position matrix and CNN for time series classification[J]. Complex & Intelligent Systems, 2021, 7(3): 1619-1631.
[9] DE SANTANA CORREIA A, COLOMBINI E L. Attention, please! A survey of neural attention models in deep learning[J]. Artificial Intelligence Review, 2022, 55: 6037-6124.
[10] TAY Y, DEHGHANI M, BAHRI D, et al. Efficient transformers: a survey[J]. ACM Computing Surveys, 2022, 55(6): 109.
[11] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30, 2017.
[12] YANG Z, DAI Z, YANG Y, et al. Xlnet: generalized autoregressive pretraining for language understanding[C]//Advances in Neural Information Processing Systems 32, 2019.
[13] SHAW P, USZKOREIT J, VASWANI A. Self-attention with relative position representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2, Short Papers), 2018: 464-468.
[14] HE P, LIU X, GAO J, et al. DEBERTA: decoding-enhanced BERT with disentangled attention[C]//Proceedings of the 2020 International Conference on Learning Representations, 2020.
[15] KE G, HE D, LIU T Y. Rethinking positional encoding in language pre-training[C]//Proceedings of the 2020 International Conference on Learning Representations, 2020.
[16] GOODFELLOW I, BENGIO Y, COURVILLE A. Deep learning[M]. Cambridge: MIT Press, 2016.
[17] YU F, KOLTUN V, FUNKHOUSER T. Dilated residual networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, 2017: 472-480.