Abstract:

The end-to-end dialogue system based on deep learning has become hotspot research in academia and industry, because it has the advantages of strong generalization ability, few training parameters, and good performance. The results of intent detection and semantic slot filling are critical to the performance of the dialogue system. This paper introduces the mainstream methods of joint intent detection and semantic slot filling in an end-to-end task-oriented dialogue system. It not only summarizes the advantages of attention mechanism and Transformer model compared to recurrent neural network and long short-term memory network in capturing long-term dependencies, but also introduces the problem of imperfect capture of word position information caused by parallel processing. Then, it analyzes the improvement of capsule neural networks capturing small probability semantic information and keeping feature integrity compared to convolution neural networks. Furthermore, it mainly introduces the joint recognition method based on the BERT（Bidirectional Encoder Representations from Transformers） model, which can not only process in parallel but also solve the problem of polysemy, which is the best method at present. Finally it discusses and analyzes the future research direction.

Key words: intent detection, semantic slot filling, joint recognition, Bidirectional Encoder Representations from Transformers（BERT） model, polysemy

摘要：

目前基于深度学习的端到端对话系统因具有泛化能力强、训练参数少、性能好等优势，在学术界和工业界成为了研究热点。意图识别和语义槽填充的结果对于对话系统的性能至关重要。介绍了端到端任务型对话系统意图和语义槽联合识别的主流方法，对注意力机制、Transformer模型在捕获长期依赖关系方面的效果同循环神经网络、长短时记忆网络进行对比，并分析了因其并行处理导致无法对文本词序位置信息完整捕获的局限；阐述了胶囊网络相较于卷积神经网络在捕获小概率语义信息保证特征完整性方面的优势；重点介绍了基于BERT（Bidirectional Encoder Representations from Transformers）模型的联合识别方法，不仅能够并行处理而且可以解决一词多义的问题，是目前性能最好的方法。最后对未来研究的发展方向进行讨论和分析。

关键词: 意图识别, 语义槽填充, 联合识别, BERT模型, 一词多义