关键词: 云计算, hadoop, mapreduce, 并行计算, 数据挖掘, 随机抽样

Abstract: In today’s “information explosion” society, data mining, because of mass data, faces a new challenges. When data mining turns to cloud computing platform to realize parallel, the study of parallel data random sampling further reduces the size of the data size. This paper presents a mapreduce parallel sampling algorithm which not only can clean up dirty data but also achieves the goal of equal probability sampling. The algorithm just needs to scan processed data only one time. It runs this algorithm in the hadoop platform and compares its performance with common random sampling. As a result, this new algorithm obtains a very high time efficiency. It is a kind of effective method which lays a good foundation for doing research on sampling in future. It can also promote data mining in the condition of facing mass data.

Key words: cloud computing, hadoop, mapreduce, parallel computing, data mining, random sampling