关键词: 文本挖掘, 短文本聚类, K-means算法, 最大频繁词集, 知网, 语义相似度

Abstract: Nowadays, there are three major challenges for short text clustering, the sparsity of feature key, the complexity of processing in high-dimensional space and the comprehensibility of clusters. For these challenges, a K-means clustering algorithm is proposed, which is improved by combining with semantic. Short text is described by collection of words in this algorithm, it alleviates the sparsity problem of characteristics of short text keywords. The clustering center can be obtained by mining the maximum frequent word set of short text collection, which effectively overcomes the defect that K-means clustering algorithm is sensitive to the initial clustering center, it solves the problem of the comprehensibility of clusters, and avoids the operation in high-dimensional space. The experimental results show that short text clustering algorithm combined with semantic is better than traditional algorithms.

Key words: text mining, clustering of short text, K-means algorithm, maximum frequent word set, HowNet, semantic similarity