计算机工程与应用 ›› 2022, Vol. 58 ›› Issue (3): 165-171.DOI: 10.3778/j.issn.1002-8331.2008-0151

• 模式识别与人工智能 • 上一篇    下一篇

结合多头注意力机制的旅游问句分类研究

吴迪,姜丽婷,王路路,吐尔根·依布拉音,艾山·吾买尔,早克热·卡德尔   

  1. 1.新疆大学 软件学院,乌鲁木齐 830046
    2.新疆大学 信息科学与工程学院,乌鲁木齐 830046
  • 出版日期:2022-02-01 发布日期:2022-01-28

Research on Classification of Tourist Questions Combined with Multi-head Attention Mechanism

WU Di, JIANG Liting, WANG Lulu, Tuergen Yibulayin, Aishan Wumaier, Zaokere Kadder   

  1. 1.College of Software, Xinjiang University, Urumqi 830046, China
    2.College of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
  • Online:2022-02-01 Published:2022-01-28

摘要: 旅游问句具有长度较短,不严格按照语法规则的特点,导致该文本数据信息容量过少、口语化严重。充分理解问句表达的语义是提高旅游问句分类器性能面临的重要挑战,基于此,提出一个融合Bi-GRU、CNN与Multi-Head-Attention的旅游问句分类模型。该模型将预先训练的词向量和经Bi-GRU处理得到的语义信息进行融合,进行问句依赖关系学习,通过CNN和Multi-Head-Attention进行特征提取,以加强局部特征的学习,通过Softmax完成分类。实验结果表明,该模型在文本信息少、表述不规范的旅游问句分类任务中F1值达到了92.11%,优于现有的主流分类模型。

关键词: 自然语言处理, 旅游问句分类, 双向门控循环单元(Bi-GRU), 卷积神经网络(CNN), 多头注意力机制

Abstract: Tourism questions have the characteristics of short length and not strictly following grammatical rules, which leads to too little information capacity of the text data and serious colloquial. It is an important challenge to fully understand the semantics of question expression to improve the performance of tourist question classifiers. To this end, a tourist question classification model combining Bi-GRU, CNN, and Multi-Head-Attention is proposed. The model fuses pre-trained word vectors and semantic information processed by Bi-GRU to learn question dependency, then extracts features through CNN and Multi-Head-Attention to strengthen the learning of local features, and finally completes classification through Softmax. The F1 score of this model achieves 92.11% in the classification task of tourism questions with less text information and irregular expression, which is superior to the existing mainstream classification model.

Key words: natural language processing, tourism question classification, bidirectional gated recurrent unit(Bi-GRU), convolutional neural network(CNN), Multi-Head-Attention