Abstract: Recurrent neural networks are widely used in various sequence data processing tasks, such as machine translation, speech recognition, image annotation and so on. The language model based on recurrent neural networks usually contains a large number of parameters, which limits the use of the model on mobile devices or embedded devices to some extent. Aiming at this problem, a low rank reconstruction compression method based on time-error is proposed, which adds the time-error reconstruction function on the basis of low rank reconstruction compression, and the input activation mechanism of long short-term memory network is adopted. Numerical experiments on multiple data sets show that the proposed method has a better effect on compression.

Key words: recurrent neural networks, long short-term memory, low rank reconstruction compression, low rank reconstruction compression based on time-error

摘要： 循环神经网络被广泛应用于各种序列数据处理任务中，如机器翻译、语音识别、图像标注等。基于循环神经网络的语言模型通常包含大量的参数，这一点在一定程度上限制了模型在移动设备或嵌入式设备上的使用。在低秩重构压缩的基础上，增加时间误差重构函数，并采用长短时记忆网络中的输入激活机制，提出了一种基于时间误差的低秩重构压缩方法。多个数据集上的数值实验表明，该方法具有较好的压缩效果。

关键词: 循环神经网络, 长短时记忆网络, 低秩重构压缩, 基于时间误差的低秩重构压缩