关键词: Storm, KNN算法, 流数据, 大数据, 数据划分

Abstract: KNN（K-Nearest Neighbor） algorithm is a kind of classification algorithm which is simpler, more effective and easier to implement. It can be applied in the classification for larger data domain. In recent years, KNN algorithm has been paid great attention to study static big data sets, however, KNN algorithm has to be processed the streaming data sets online in more and more scenarios. Considering the streaming data with the characteristics of large, continuous, fast, not easy to store and restore; and the streaming processing system Storm with the characteristics of real-time and reliability, a modified KNN is proposed, which implements KNN on Strom to classify the streaming data online. By partitioning the whole sample set into multiple piece sets first, it then computes KNN of those to-be-classified vectors on each piece set, finally, the KNN are reduced to the whole KNN, thereby to achieve the classification of the to-be-classified vectors. Experiment results show that the proposed algorithm is able to meet the requirements of high throughput, scalability, real-time and accuracy for the classification of streaming data on the big data background.

Key words: Storm, K-Nearest Neighbor（KNN）, streaming data, big data, data partition