计算机工程与应用 ›› 2014, Vol. 50 ›› Issue (2): 129-132.

• 数据库、数据挖掘、机器学习 • 上一篇    下一篇

一种融合词语位置特征的Lucene相似度评分算法

白培发1,王成良1,2,徐  玲2   

  1. 1.重庆大学 计算机学院,重庆 400030
    2.重庆大学 软件学院,重庆 400030
  • 出版日期:2014-01-15 发布日期:2014-01-26

Scoring algorithm of similarity based on terms’ position feature combination for Lucene

BAI Peifa1, WANG Chengliang1,2, XU Ling2   

  1. 1.College of Computer Science, Chongqing University, Chongqing 400030, China
    2.College of Software Engineering, Chongqing University, Chongqing 400030, China
  • Online:2014-01-15 Published:2014-01-26

摘要: 相似度评分算法是Lucene引擎中的核心部分之一。对Lucene内部的相似度评分算法进行研究分析后,针对Lucene只关心查询词出现的频率,而不关心它们所在的位置这一缺陷提出了一种改进的算法。改进的算法将词语位置关系特征融合到Lucene原始相似度评分算法中。在TREC数据集上的实验结果表明:改进后的算法与Lucene原始算法相比,在MAP和P@n指标上都有一定程度的提高。

关键词: Lucene, 相似度, 全文检索

Abstract: The scoring algorithm of similarity is one of the core parts in Lucene. After the analysing and researching on the default scoring algorithm of Lucene similarity, this paper proposes an improved algorithm aimed at the deficiency of the Lucene’s default algorithm which only considers the frequencies rather than the position of query terms occurrence. The improved algorithm combines the feature of the terms’ position relationship with Lucene’s default scoring algorithm of similarity. The experiment on the TREC dataset shows that, the improved algorithm increases the value of evaluation metric MAP and P@n to a certain extent.

Key words: Lucene, similarity, full text search