Computer Engineering and Applications ›› 2011, Vol. 47 ›› Issue (33): 156-158.

• 数据库、信号与信息处理 • Previous Articles     Next Articles

Chinese short text clustering based on dynamic vector

JIN Chunxia,ZHOU Haiyan   

  1. Faculty of Computer Engineering,Huaiyin Institute of Technology,Huai’an,Jiangsu 223003,China
  • Received:1900-01-01 Revised:1900-01-01 Online:2011-11-21 Published:2011-11-21

动态向量的中文短文本聚类

金春霞,周海岩   

  1. 淮阴工学院 计算机工程学院,江苏 淮安 223003

Abstract: Since Chinese short text is short of keywords and full of anomalous words,it brings about short text similarity drift and the traditional text clustering method is not directly suitable for short text clustering.To solve the problem of sparse key-words and similarity drift in short text segments,this paper proposes a new method to build dynamic text vector by text similarity based on HowNet.This method can measure the similarity between short text segments by dynamic text vector,then find short text relationship,so as to relieve these two characteristics’ bad influence on the clustering performance and therefore to gain a better clustering result.Experiments show the method can get better performance in Chinese short text clustering,compared with traditional method.

Key words: short text, similarity between short text segments, dynamic vector, text clustering algorithm, K-means algorithm

摘要: 因中文短文本特征词词频低、存在大量变形词和新词的特点,使得中文短文本相似度发生漂移,难以直接使用现有的面向长文本的聚类算法。针对短文本相似度漂移问题,提出了一种基于《知网》扩充相关词集构建动态文本向量的方法,利用动态向量计算中文短文本的内容相似度,进而发现短文本之间的内在关联,从而缓解特征词词频过低和存在变形词以及新词对聚类的影响,获得较好的聚类结果。实验结果表明,该算法的聚类质量高于传统算法。

关键词: 短文本, 文本相似度, 动态表示向量, 文本聚类, K-means算法