Answer Selection Model for Multi-Stage Matching

doi:10.3778/j.issn.1002-8331.2108-0354

Abstract

Abstract: In recent years, the cost of obtaining effective information is higher and higher. Answer selection（AS） can directly provide users with the information they need, which has revolutionary significance. Given a question and a list of candidate answers, AS requires to find the most relevant answer from the candidate answers. Without loss of generality, candidate answers can be divided into three types according to the degree of matching with the question：irrelevant, relevant and unreasonable, relevant and reasonable. However, the existing work only considers the relevance between questions and answers. Therefore, this paper proposes a multi-stage matching model（MSMM） to simulate the answering process of human. Specifically, the MSMM model is divided into two stages. In the first stage, the easy samples are separated, and in the second stage, the complex samples are synthesized. Each stage consists of embedding layer, encoding layer, alignment layer, fusion layer, and pooling layer. In addition, to enhance the reasoning ability of the model, this paper also introduces semantic role labeling and word similarity matrix. To facilitate the evaluation, this paper builds a benchmark based on WikiQA and InsuranceQA datasets. Empirical studies demonstrate that the proposed model can achieve consistent performance improvement compared with baselines.

Key words: answer selection, question answering, knowledge enhancement

摘要： 近年来，信息量成倍增长，获取有效信息的代价越来越高，答案选择技术能够为用户直接提供所需的信息，具有革命性的意义。给定问题和候选答案，答案选择任务要求从候选答案中找出与问题最相关的答案。不失一般性，候选答案根据与问题的匹配程度可以分为三种类型：不相关、相关不合理、相关且合理。然而，已有工作仅考虑问题与答案的相关性，这对于精准问答是远远不够的。为此，提出多阶段匹配模型（MSMM），模拟人的答题过程。具体的，MSMM模型分为两个阶段，第一阶段先将简单易解决的问答对分离出去，第二阶段再综合推理复杂的问答数据。每一阶段都由嵌入层、编码层、对齐层、融合层和池化层组成。此外，为了增强模型的推理能力，还引入语义角色标注信息和单词相似矩阵信息。为了便于评估，基于WikiQA和InsuranceQA数据集构造了两个答案合理性数据集。实验结果表明，对比基准方法，该模型在性能上取得一致的提升。

关键词: 答案选择, 问答系统, 知识增强

CHEN Jiangui, ZHANG Ruqing, GUO Jiafeng, FAN Yixing. Answer Selection Model for Multi-Stage Matching[J]. Computer Engineering and Applications, 2023, 59(3): 158-165.

陈建贵, 张儒清, 郭嘉丰, 范意兴. 面向多阶段匹配的答案选择模型[J]. 计算机工程与应用, 2023, 59(3): 158-165.

References

[1] LASKAR M T R，HUANG X，HOQUE E.Contextualized embeddings based transformer encoder for sentence similarity modeling in answer selection task[C]//Proceedings of the 12th Language Resources and Evaluation Conference，2020：5505-5514.
[2] YANG L，AI Q，GUO J，et al.aNMM：ranking short answer texts with attention-based neural matching model[C]//Proceedings of the 25th ACM International on Conference on Information and Knowledge Management，2016：287-296.
[3] YANG Y，YIH W，MEEK C.WikiQA：a challenge dataset for open-domain question answering[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing，2015：2013-2018.
[4] FENG M，XIANG B，GLASS M R，et al.Applying deep learning to answer selection：a study and an open task[C]//2015 IEEE Workshop on Automatic Speech Recognition and Understanding（ASRU），2015：813-820.
[5] CHEN Q，HU Q，HUANG J X，et al.CA-RNN：using context-aligned recurrent neural networks for modeling sentence similarity[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[6] TAY Y，TUAN L A，HUI S C.Multi-cast attention networks[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining，2018：2299-2308.
[7] TRAN Q H，LAI T，HAFFARI G，et al.The context-dependent additive recurrent neural net[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies（Volume 1：Long Papers），2018：1274-1283.
[8] YOON S，DERNONCOURT F，KIM D S，et al.A compare-aggregate model with latent clustering for answer selection[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management，2019：2093-2096.
[9] WANG M，SMITH N A，MITAMURA T.What is the Jeopardy model? A quasi-synchronous grammar for QA[C]//Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning（EMNLP-CoNLL），2007：22-32.
[10] KO J，MITAMURA T，NYBERG E.Language-independent probabilistic answer ranking for question answering[C]//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics，2007：784-791.
[11] YIH W，CHANG M W，MEEK C，et al.Question answering using enhanced lexical semantic models[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2013：1744-1753.
[12] RIEZLER S，VASSERMAN A，TSOCHANTARIDIS I，et al.Statistical machine translation for query expansion in answer retrieval[C]//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics，2007：464-471.
[13] SHEN G，YANG Y，DENG Z H.Inter-weighted alignment network for sentence pair modeling[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing，2017：1179-1189.
[14] TRAN Q H，LAI T，HAFFARI G，et al.The context-dependent additive recurrent neural net[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies（Volume 1：Long Papers），2018：1274-1283.
[15] SHEN Y，HE X，GAO J，et al.Learning semantic representations using convolutional neural networks for web search[C]//Proceedings of the 23rd International Conference on World Wide Web，2014：373-374.
[16] MADABUSHI H T，LEE M，BARNDEN J.Integrating question classification and deep learning for improved answer selection[C]//Proceedings of the 27th International Conference on Computational Linguistics，2018：3283-3294.
[17] KAMATH S，GRAU B，MA Y.Predicting and integrating expected answer types into a simple recurrent neural network model for answer sentence selection[J].Computacióny Sistemas，2019，23（3）.
[18] GARG S，VU T，MOSCHITTI A.Tanda：transfer and adapt pre-trained transformer models for answer sentence selection[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：7780-7788.
[19] SUN C，QIU X，XU Y，et al.How to fine-tune BERT for text classification?[C]//China National Conference on Chinese Computational Linguistics.Cham：Springer，2019：194-206.
[20] YANG W，XIE Y，LIN A，et al.End-to-end open-domain question answering with bertserini[J].arXiv：1902.01718，2019.
[21] YANG R，ZHANG J，GAO X，et al.Simple and effective text matching with richer alignment features[J].arXiv：1908.00300，2019.
[22] ZHANG Z，WU Y，ZHAO H，et al.Semantics-aware BERT for language understanding[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：9628-9635.
[23] XIA T，WANG Y，TIAN Y，et al.Using prior knowledge to guide BERT’s attention in semantic textual matching tasks[C]//Proceedings of the Web Conference，2021：2466-2475.
[24] LIU W，ZHOU P，ZHAO Z，et al.Fastbert：a self-distilling bert with adaptive inference time[J].arXiv：2004.02178，2020.
[25] PENNINGTON J，SOCHER R，MANNING C D.Glove：global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing（EMNLP），2014：1532-1543.
[26] KINGMA D P，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.
[27] PETERS M E，NEUMANN M，IYYER M，et al.Deep contextualized word representations[J].arXiv：1802.05365，2018.
[28] PALMER M，GILDEA D，KINGSBURY P.The proposition bank：an annotated corpus of semantic roles[J].Computational Linguistics，2005，31（1）：71-106.
[29] TAN M，DOS SANTOS C，XIANG B，et al.Improved representation learning for question answer matching[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics（Volume 1：Long Papers），2016：464-473.
[30] CHEN Q，ZHU X，LING Z，et al.Enhanced LSTM for natural language inference[J].arXiv：1609.06038，2016.
[31] SHEN Y，DENG Y，YANG M，et al.Knowledge-aware attentive neural network for ranking question answer pairs[C]//The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval，2018：901-904.
[32] DENG Y，XIE Y，LI Y，et al.Contextualized knowledge-aware attentive neural network：enhancing answer selection with knowledge[J].arXiv：2104.05216，2021.