Abstract: End-to-end neural networks can automatically learn feature transformation from original data, which can solve the mismatch between hand designed features and specific tasks. The traditional end-to-end neural network for speech recognition uses a time domain convolution network as the feature extraction model, recurrent neural network and full connected feed-forwarddeep neural network as the acoustic model, which has some limitations in performance and efficiency. From the aspects of the performanceof thefeature extraction module and the training efficiency of the acoustic model, an end-to-end speech recognition model combining the multi-time and frequency resolution convolution and the feed-forward neural network with memory modules is proposed. On the real recording test dataset, the proposed method reduces the word error rate by 10%, training time by 80% compared with the traditional method.

Key words: convolution neural network, recurrent neural network, acoustic model, end-to-end neural network

摘要： 端到端神经网络能够根据特定的任务自动学习从原始数据到特征的变换，解决人工设计的特征与任务不匹配的问题。以往语音识别的端到端网络采用一层时域卷积网络作为特征提取模型，递归神经网络和全连接前馈深度神经网络作为声学模型的方式，在效果和效率两个方面具有一定的局限性。从特征提取模块的效果以及声学模型的训练效率角度，提出多时间频率分辨率卷积网络与带记忆模块的前馈神经网络相结合的端到端语音识别模型。实验结果表明，所提方法语音识别在真实录制数据集上较传统方法字错误率下降10%，训练时间减少80%。

关键词: 卷积神经网络, 递归神经网络, 声学模型, 端到端模型