Intent Detection of Domain Adaptation Combined with Capsule Network

doi:10.3778/j.issn.1002-8331.2102-0235

Abstract

Abstract:

Intent detection is an important task in spoken language understanding, which is related to the performance of the entire dialogue system. Aiming at the problem of less training corpus in the human-machine dialogue system in the new domain, and the construction of training corpus is very expensive. This thesis proposes a domain adaptation method using capsule network to improve the domain discriminator. This method uses a domain adversarial neural network to transfer the feature information of the source domain to the target domain, in addition, in order to ensure the feature quality of the domain intent text, the feature representations of the source domain and the target domain are extracted again, which can fully obtain the feature information of the intent text, captures the unique features of different domains, improves the discriminator ability of the domain, and ensures the reliability of the guarantee domain adaptation tasks. When the target domain contains only a small number of labeled samples, the accuracy rate on Chinese and English datasets reaches 83.3% and 88.9%.

Key words: intent detection, dialogue system, capsule network, domain adaptation

摘要：

意图识别是口语理解中的重要任务，关乎整个对话系统的性能。针对新领域人机对话系统中训练语料较少，构建可训练语料十分昂贵的问题，提出一种利用胶囊网络改进领域判别器的领域适应方法。该方法利用领域对抗神经网络将源域的特征信息迁移至目标域中，此外，为了保证领域意图文本的特征质量，对源域和目标域的特征表示进行再次提取，充分获取意图文本的特征信息，捕捉不同领域的独有特征，提高领域的判别能力，保障领域适应任务的可靠性。在目标域仅包含少量样本的情况下，该方法在中文和英文数据集上的准确率分别达到了83.3%和88.9%。

关键词: 意图识别, 对话系统, 胶囊网络, 领域适应

ZHAO Pengfei, LI Yanling, LIN Min. Intent Detection of Domain Adaptation Combined with Capsule Network[J]. Computer Engineering and Applications, 2021, 57(21): 188-194.

赵鹏飞，李艳玲，林民. 结合胶囊网络的领域适应意图识别[J]. 计算机工程与应用, 2021, 57(21): 188-194.

References

[1] 赵鹏飞，李艳玲，林民.面向迁移学习的意图识别研究进展[J].计算机科学与探索，2020，14（8）：1261-1274.
ZHAO P F，LI Y L，LIN M.Research progress on intent detection oriented to transfer learning[J].Journal of Frontiers of Computer Science and Technology，2020，14（8）：1261-1274.
[2] WU Y，ZHANG Z，WANG G.Unsupervised deep feature transfer for low resolution image classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops，2019.
[3] CHEN E，MATHAI T S，SARODE V，et al.A study of domain generalization on ultrasound-based multi-class segmentation of arteries，veins，ligaments，and nerves using transfer learning[J].arXiv：2011.07019，2020.
[4] 杨志明，王来奇，王泳.基于双通道卷积神经网络的问句意图分类研究[J].中文信息学报，2019，33（5）：122-131.
YANG Z M，WANG L Q，WANG Y.Questions intent classification based on dual channel convolutional neural network[J].Journal of Chinese Information Processing，2019，33（5）：122-131.
[5] 杨志明，王来奇，王泳.深度学习算法在问句意图分类中的应用研究[J].计算机工程与应用，2019，55（10）：154-160.
YANG Z M，WANG L Q，WANG Q.Application research of deep learning algorithm in questions intention classification[J].Computer Engineering and Applications，2019，55（10）：154-160.
[6] 侯丽仙，李艳玲，林民，等.融合多约束条件的意图和语义槽填充联合识别[J].计算机科学与探索，2020，14（9）：1545-1553.
HOU L X，LI Y L，LIN M，et al.Joint Recognition of intent and semantic slot filling combining multiple constraints[J].Journal of Frontiers of Computer Science and Technology，2020，14（9）：1545-1553.
[7] 刘娇，李艳玲，林民.胶囊网络用于短文本多意图识别的研究[J].计算机科学与探索，2020，14（10）：1735-1743.
LIU J，LI Y L，LIN M.Capsule network is used in the research of short text multi-intent detection[J].Journal of Frontiers of Computer Science and Technology，2020，14（10）：1735-1743.
[8] 尹春勇，何苗.基于改进胶囊网络的文本分类[J].计算机应用，2020，40（9）：2525-2530.
YIN C Y，HE M.Text classification based on improved capsule network[J].Journal of Computer Applications，2020，40（9）：2525-2530.
[9] PAN S J，TSANG I W，KWOK J T.et al.Domain adaptation via transfer component analysis[J].IEEE Transactions on Neural Network，2010，22（2）：199-210.
[10] ZELLINGER W，GRUBINGER T，LUGHOFER E，et al.Central moment discrepancy（cmd） for domain-invariant representation learning[J].arXiv：1702.08811，2017.
[11] GOODFELLOW I J，JEAN P，MIRZA M，et al.Generative adversarial networks[J].arXiv：1406.2261，2014.
[12] AJAKAN H，GERMAIN P，LAROCHELLE H，et al.Domain-adversarial neural networks[J].arXiv：1412.4446，2014.
[13] GANIN Y，USTINOVA E，AJAKAN H，et al.Domain- adversarial training of neural network[J].arXiv：1505. 07818，2015.
[14] GRIEHABER D，VU N T，MAUCHER J.Low-resource text classification using domain-adversarial learning[J].arXiv：1807.05195v2，2020.
[15] COUCKE A，SAADE A，BALL A，et al.Snips voice platform：an embedded spoken language understanding system for private-by-design voice interfaces[J].arXiv：1805.10190，2018.
[16] HEMPHILL C T，GODFREY J J，DODDINGTON G R.The ATIS spoken language systems pilot corpus[C]//Speech and Natural Language：Proceedings of a Workshop Held at Hidden Valley，Pennsylvania，June 24-27，1990.
[17] ZHANG W N，CHEN Z G，CHE W X，et al.The first evaluation of chinese human-computer dialogue technology[J].arXiv：1709.10217，2017.
[18] TZENG E，HOFFMAN J，ZHANG N，et al.Deep domain confusion：maximizing for domain invariance[J].arXiv：1412.3474，2014.