计算机工程与应用 ›› 2013, Vol. 49 ›› Issue (14): 117-120.

• 数据库、数据挖掘、机器学习 • 上一篇    下一篇

一种基于数据压缩的Apriori算法

高海洋1,2,沈  强1,张轩溢1,赵志军1   

  1. 1.中国科学院 声学研究所 高性能网络实验室,北京 100190
    2.无锡中科智能信息处理研发中心有限公司,江苏 无锡 214135
  • 出版日期:2013-07-15 发布日期:2013-07-31

Improved Apriori based on data compression

GAO Haiyang1,2, SHEN Qiang1, ZHANG Xuanyi1, ZHAO Zhijun1   

  1. 1.High Performance Network Lab, Institute of Acoustics, China Academy of Sciences, Beijing 100190, China
    2.Wuxi Zhongke R&D Center of Intelligent Information Processing Technologies, Wuxi, Jiangsu 214135, China
  • Online:2013-07-15 Published:2013-07-31

摘要: 随着物联网技术的飞速发展,数据采集手段迅速增加,对海量数据分析与处理的需求也愈加强烈。关联规则挖掘算法通过数据之间的关联分析,挖掘出数据之间的隐含关系,进而获得了大量应用。在众多的关联规则算法中,传统的Apriori算法虽然得到了大量应用,但是因为该算法产生大量的候选集,而且需要多次对数据库进行扫描,导致该算法的运行效率大大降低。为了克服Apriori算法的以上缺点,通过数据压缩的方法减少了数据库扫描次数的同时,对生成的候选集进行了多次验证,大大减少了无效候选集的数量。大量的数据挖掘实验证明提出的改进算法可以在正确挖掘数据集关联规则的同时,大大提高了算法的运行效率。

关键词: 数据挖掘, 关联规则, Apriori算法, 数据压缩, 频繁集检测

Abstract: The Apriori algorithm is one of the most influential algorithms for mining association rules. It can work on the large dataset efficiently. However, the traditional?Apriori algorithm?has two?bottlenecks. It generates a?large number of?candidate?sets, and most of them are useless. It has to scan?the database for many times. This paper presents an improved Apriori algorithm based on the data compression methodology. The improved algorithm can reduce the number of database scans and the number of candidate set by pre-judging at the same time. Complicated experiment demonstrates that a significant improvement has been achieved by the algorithm.

Key words: data mining, association rules, Apriori, data compression, detection of frequent set