Retrieval?Augmented Question?Answering Generation Based on Question?Oriented Prompt Learning and Multi-Channel Reasoning

doi:10.3778/j.issn.1002-8331.2409-0336

Abstract

Abstract: The current large language models have important applications in various fields due to their superior performance. However, large language models have problems such as outdated knowledge, insufficient knowledge, and incorrect generated results. Retrieval augmented generation alleviates these issues by providing accurate and timely professional external knowledge input to large language models. However, how to improve the accuracy of generated answers is still a hot problem worth paying attention to. This paper designs question-oriented prompts and optimizes the prompt engineering to effectively stimulate the understanding ability of large language models for different types of questions, and fully utilizes external information to improve the accuracy of retrieval augmented generation in answering question and answer tasks for different types of questions. This paper uses auxiliary multi-way reasoning to optimize prompts and generate answer summaries to reduce the illusion of model generated answers. The experimental results on simple questions, comparison questions, set questions, multi-hop questions, and total data in the English retrieval augmented generation benchmark dataset show that the proposed method can achieve good implementation results.

Key words: retrieval-augmented generation, prompt learning, large language model

摘要： 当前大语言模型因其优越的性能，在各个领域都有着重要的应用。但大语言模型存在知识过时、知识不充分及生成结果错误等问题。检索增强生成通过给大语言模型输入精准及时的专业外部知识缓解了这些问题。然而，如何提高生成答案的准确性仍是值得关注的热点问题。设计问题导向提示，通过优化提示工程，有效激发了大语言模型对于不同类型问题的理解能力，并充分利用外部信息，提高检索增强生成在回答不同类型问题的问答任务的准确性。同时使用辅助多路推理优化提示与生成答案总结降低模型生成答案的幻觉。在英文检索增强生成基准数据集中的简单问题、比较问题、集合问题与多跳问题以及总体数据进行的实验结果表明，提出的方法能够取得比较好的实现效果。

关键词: 检索增强生成, 提示学习, 大语言模型

WANG Yuting, CHEN Bo, YAN Qiang, FAN Yixing, YU Zhihua, GUO Jiafeng. Retrieval?Augmented Question?Answering Generation Based on Question?Oriented Prompt Learning and Multi-Channel Reasoning[J]. Computer Engineering and Applications, 2025, 61(12): 120-128.

王昱婷, 陈波, 闫强, 范意兴, 余智华, 郭嘉丰. 基于问题导向提示学习和多路推理的检索增强生成问答[J]. 计算机工程与应用, 2025, 61(12): 120-128.

References

[1] TOUVRON H, LAVRIL T, IZACARD G, et al. LLaMA: open and efficient foundation language models[J]. arXiv: 2302.13971, 2023.
[2] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[J]. Computer Science, 2018: 49313245.
[3] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[4] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017: 5998-6008.
[5] LEWIS P, PEREZ E, PIKTUS A, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks[C]//Advances in Neural Information Processing Systems, 2020: 9459-9474.
[6] GAO Y F, XIONG Y, GAO X Y, et al. Retrieval-augmented generation for large language models: a survey[J]. arXiv:2312.10997, 2023.
[7] YANG X, SUN K, XIN H, et al. CRAG: comprehensive RAG benchmark[J]. arXiv:2406.04744, 2024.
[8] LIU P F, YUAN W Z, FU J L, et al. Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing[J]. ACM Computing Surveys, 2023, 55(9): 1-35.
[9] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[10] COLIN R, NOAM S, ADAM R, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of Machine Learning Research, 2020, 21(1): 5485-5551.
[11] BROWN T, MANN B, RYDER N, et al. Language models are few-shot learners[C]//Advances in Neural Information Processing Systems, 2020: 1877-1901.
[12] ACHIAM J, ADLER S, AGARWAL S, et al. GPT-4 technical report[J]. arXiv:2303.08774, 2023.
[13] ROBERTSON S E, WALKER S. On relevance weights with little relevance information[C]//Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval-SIGIR’97. New York: ACM, 1997: 16-24.
[14] ROBERTSON S, ZARAGOZA H. The probabilistic relevance framework: bm25 and beyond[J]. Foundations and Trends? in Information Retrieval, 2009, 3(4): 333-389.
[15] KARPUKHIN V, O?UZ B, MIN S, et al. Dense passage retrieval for open-domain question answering[J]. arXiv: 2004.04906, 2020.
[16] ASAI A, WU Z Q, WANG Y Z, et al. Self-RAG: learning to retrieve, generate, and critique through self-reflection[J]. arXiv:2310.11511, 2023.
[17] WANG L, YANG N, WEI F R. Query2doc: query expansion with large language models[J]. arXiv:2303.07678, 2023.
[18] MA X B, GONG Y Y, HE P C, et al. Query rewriting for retrieval-augmented large language models[J]. arXiv:2305. 14283, 2023.
[19] SHI W J, MIN S, YASUNAGA M, et al. REPLUG: retrieval-augmented black-box language models[J]. arXiv:2301. 12652, 2023.
[20] ZHANG Y X, KHALIFA M, LOGESWARAN L, et al. Merging generated and retrieved knowledge for open-domain QA[J]. arXiv:2310.14393, 2023.
[21] 田永林, 王兴霞, 王雨桐, 等. RAG-PHI: 检索增强生成驱动的平行人与平行智能[J]. 智能科学与技术学报, 2024, 6(1): 41-51.
TIAN Y L, WANG X X, WANG Y T, et al. RAG-PHI: rag-driven parallel human and parallel intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2024, 6(1): 41-51.
[22] 刘彦宏, 崔永瑞. 基于Word2Vec模型与RAG框架的医疗检索增强生成算法[J]. 人工智能与机器人研究, 2024(3): 479-486.
LIU Y H, CUI Y R. Enhanced generation algorithm for medical retrieval based on Word2Vec model and RAG framework[J]. Artificial Intelligence and Robotics Research, 2024(3): 479-486.
[23] KIM H J, CHO H, KIM J, et al. Self-generated in-context learning: leveraging auto-regressive language models as a demonstration generator[J]. arXiv:2206.08082, 2022.
[24] RAM O, LEVINE Y, DALMEDIGOS I, et al. In-context retrieval-augmented language models[J]. Transactions of the Association for Computational Linguistics, 2023, 11: 1316-1331.
[25] YU W H, ITER D, WANG S H, et al. Generate rather than retrieve: large language models are strong context generators[J].arXiv:2209.10063, 2022.
[26] KANG B, KIM J, YUN T R, et al. Prompt-RAG: pioneering vector embedding-free retrieval-augmented generation in niche domains, exemplified by Korean medicine[J].arXiv:2401.11246, 2024.
[27] WANG Y Q, YAO Q M, KWOK J T, et al. Generalizing from a few examples[J]. ACM Computing Surveys, 2021, 53(3): 1-34.
[28] WANG P F, WANG Z Y, LI Z, et al. SCOTT: self-consistent chain-of-thought distillation[J]. arXiv:2305.01879, 2023.