基于多表达的第一阶段语义检索模型

doi:10.3778/j.issn.1002-8331.2108-0343

摘要/Abstract

摘要： 当前，信息检索系统通常采用“检索+重排序”的多级流水线架构。基于稠密表示的检索模型已经被逐渐应用到第一阶段检索中，并展现出了相比传统的稀疏向量空间模型更好的性能。考虑到第一阶段检索所需的高效性，大多数情况下这些模型的基本架构都采用双编码器（bi-encoder）结构。对查询和文档进行独立的编码，分别得到一个稠密表示向量，然后基于获得的查询和文档表示使用简单的相似度函数计算查询-文档对的得分。然而，在编码文档的过程中查询是不可知的，而且文档相比查询而言通常包含更多的主题信息，因此这种简单的单表示模型可能会造成严重的文档信息丢失。为了解决这个问题，设计了一种新的语义检索方法MDR（multi-representation dense retrieval），将文档编码成多个稠密向量表示。同时，该方法引入覆盖率（coverage）机制来保证多个向量之间的差异性，从而能够覆盖文档中不同主题的信息。为了评估模型性能，在MS MARCO数据集上进行了段落排序和文档排序任务，实验结果证明了MDR方法的有效性。

关键词: 语义检索, 双编码器模型, 信息检索

Abstract: Modern information retrieval systems generally use the retrieval-and-ranking multi-stage architecture. Recently, retrieval models based on dense representations have been gradually applied in the first stage of document retrieval tasks, showing better performance than the traditional sparse vector space models. Considering the efficiency requirement of the first-stage retrieval, most of these models employ the bi-encoder architecture. It encodes the query and the document independently, and obtains dense representation vectors for them respectively. Then, the score of query-document pair is calculated by using a simple similarity function based on the obtained query and document representation. However, it is query agnostic when encoding the document, and documents usually contain more topic information than queries. Thus, this simple single-representation model may cause serious document information loss. To solve this problem, this work designs a new dense retrieval method, called multi-representation dense retrieval（MDR）, to encode the document into multiple dense representation vectors. At the same time, it introduces the coverage mechanism to ensure the difference between multiple vectors, so as to cover the information of different topics in the document. Experimental results of passage ranking and document ranking tasks on MS MARCO dataset prove the effectiveness of the proposed model.

Key words: semantic retrieval, bi-encoder model, information retrieval

蔡银琼, 范意兴, 郭嘉丰, 张儒清. 基于多表达的第一阶段语义检索模型[J]. 计算机工程与应用, 2023, 59(4): 139-146.

CAI Yinqiong, FAN Yixing, GUO Jiafeng, ZHANG Ruqing. Multi-Representation Model for the First-Stage Semantic Retrieval[J]. Computer Engineering and Applications, 2023, 59(4): 139-146.

参考文献

[1] MATVEEVA I，BURGES C，BURKARD T，et al.High accuracy retrieval with multiple nested ranker[C]//Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，2006：437-444.
[2] CHEN R C，GALLAGHER L，BLANCO R，et al.Efficient cost-aware cascade ranking in multi-stage retrieval[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval，2017：445-454.
[3] LIU S，XIAO F，OU W，et al.Cascade ranking for operational e-commerce search[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining，2017：1557-1565.
[4] NGUYEN T，ROSENBERG M，SONG X，et al.MS MARCO：a human generated machine reading comprehension dataset[C]//Proceedings of the Workshop on Cognitive Computation：Integrating Neural and Symbolic Approaches 2016 Co-located with the 30th Annual Conference on Neural Information Processing Systems，2016.
[5] DIETZ L，VERMA M，RADLINSKI F，et al.TREC complex answer retrieval overview[C]//Proceedings of the 26th Text Retrieval Conference，2017.
[6] ROBERTSON S，ZARAGOZA H.The probabilistic relevance framework：BM25 and beyond[J].Foundations and Trends in Information Retrieval，2009，3（4）：333-389.
[7] ZHAO L，CALLAN J.Term necessity prediction[C]//Proceedings of the 19th ACM International Conference on Information and Knowledge Management，2010：259-268.
[8] FURNAS G W，LANDAUER T K，GOMEZ L M，et al.The vocabulary problem in human-system communication[J].Communications of the ACM，1987，30（11）：964-971.
[9] LI H，XU J.Semantic matching in search[J].Foundations and Trends in Information Retrieval，2014，7（5）：343-469.
[10] LESK M E.Word-word associations in document retrieval systems[J].American Documentation，1969，20（1）：27-38.
[11] QIU Y，FREI H P.Concept based query expansion[C]//Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，1993：160-169.
[12] LAVRENKO V，CROFT W B.Relevance-based language models[J].ACM SIGIR Forum，2017，51（2）：260-267.
[13] METZLER D，CROFT W B.A Markov random field model for term dependencies[C]//Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，2005：472-479.
[14] GAO J，NIE J Y，WU G，et al.Dependence language model for information retrieval[C]//Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，2004：170-177.
[15] XU J，LI H，ZHONG C.Relevance ranking using kernels[C]//Asia Information Retrieval Symposium.Berlin，Heidelberg：Springer，2010：1-12.
[16] DEERWESTER S，DUMAIS S T，FURNAS G W，et al.Indexing by latent semantic analysis[J].Journal of the American Society for Information Science，1990，41（6）：391-407.
[17] WEI X，CROFT W B.LDA-based document models for ad-hoc retrieval[C]//Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，2006：178-185.
[18] BERGER A，LAFFERTY J.Information retrieval as statistical translation[J].ACM SIGIR Forum，2017，51（2）：219-226.
[19] KARIMZADEHGAN M，ZHAI C X.Estimation of statistical translation models based on mutual information for ad hoc information retrieval[C]//Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval，2010：323-330.
[20] DAI Z，CALLAN J.Context-aware term weighting for first stage passage retrieval[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval，2020：1533-1536.
[21] BAI Y，LI X，WANG G，et al.SparTerm：learning term-based sparse representation for fast text retrieval[J].arXiv：2010.00768，2020.
[22] KARPUKHIN V，O?UZ B，MIN S，et al.Dense passage retrieval for open-domain question answering[J].arXiv：2004.
04906，2020.
[23] ZHAN J，MAO J，LIU Y，et al.RepBERT：contextualized text embeddings for first-stage retrieval[J].arXiv：2006.15498，
2020.
[24] JOHNSON J，DOUZE M，JéGOU H.Billion-scale similarity search with GPUs[J].IEEE Transactions on Big Data，2021，7（3）：535-547.
[25] TU Z，LU Z，LIU Y，et al.Modeling coverage for neural machine translation[J].arXiv：1601.04811，2016.
[26] XU J X，CROFT W B.Quary expansion using local and global document analysis[J].ACM SIGIR Forum，2017，51（2）：168-175.
[27] ZHAI C，LAFFERTY J.Model-based feedback in the language modeling approach to information retrieval[C]//Proceedings of the 10th International Conference on Information and Knowledge Management，2001：403-410.
[28] ABDUL-JALEEL N，ALLAN J，CROFT W B，et al.UMass at TREC 2004：novelty and HARD[R].Computer Science Department Faculty Publication Series，2004：189.
[29] VOORHEES E M.Query expansion using lexical-semantic relations[C]//Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，1994：61-69.
[30] CAO G，NIE J Y，GAO J，et al.Selecting good expansion terms for pseudo-relevance feedback[C]//Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，2008：243-250.
[31] COLLINS-THOMPSON K.Reducing the risk of query expansion via robust constrained optimization[C]//Proceedings of the 18th ACM Conference on Information and Knowledge Management，2009：837-846.
[32] HOFMANN T.Probabilistic latent semantic indexing[C]//Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval，1999：50-57.
[33] DIAZ F.Regularizing query-based retrieval scores[J].Information Retrieval，2007，10（6）：531-562.
[34] NOGUEIRA R，YANG W，LIN J，et al.Document expansion by query prediction[J].arXiv：1904.08375，2019.
[35] MUJA M，LOWE D G.Scalable nearest neighbor algorithms for high dimensional data[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2014，36（11）：2227-2240.
[36] ZHANG M，HE Y.Grip：multi-store capacity-optimized high-performance nearest neighbor search for vector search engine[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management，2019：1673-1682.
[37] DEVLIN J，CHANG M W，LEE K，et al.BERT：pre-training of deep bidirectional transformers for language understanding[J].arXiv：1810.04805，2018.
[38] DAI Z Y，CALLAN J.Context-aware document term weighting for ad-hoc search[C]//Proceedings of the Web Conference 2020，2020：1897-1907.
[39] YANG P L，FANG H，LIN J.Anserini：enabling the use of lucene for information retrieval research[C]//Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval，2017：1253-1256.