关键词: 重复属性识别, Web搜索, 摘要, URL, 查询探针

Abstract: It needs to import data from different sources when building a data warehouse, which results in the increase of approximately duplicated records, adversely affecting the data utilization and the quality of making decisions. Therefore, detecting approximately duplicated records has become an intensive research subject. Duplicate attribute resolution is the key of approximately duplicated records detecting. This paper proposes an efficient, Web-based algorithm to automatically recognize duplicated attributes. This novel algorithm uses snippets and URL information returned by search engine to calculate attribute similarity, and further improves recognition precision by inserting query probes. The results show that the algorithm improves recall.

Key words: duplicate attribute resolution, Web search, snippet, URL, query probe