Computer Engineering and Applications ›› 2014, Vol. 50 ›› Issue (1): 1-4.

Previous Articles     Next Articles

Research on hash algorithm for distributed storage system

HUANG Qiulan, CHENG Yaodong, CHEN Gang   

  1. Computing Center, Institute of High Energy Physics, Chinese Academy of Sciences, Beijing 100049, China
  • Online:2014-01-01 Published:2013-12-30

分布式存储系统的哈希算法研究

黄秋兰,程耀东,陈  刚   

  1. 中国科学院 高能物理研究所计算中心,北京 100049

Abstract: Considering the uniform data distribution in physical storage and efficient data positioning in distributed storage system, this paper studies different hash algorithms and proposes how to measure the merits of hash algorithm in distributed storage system. Based on experiments, the detail comparative analysis of various hash algorithms are shown in aspect of hash value distribution, hash conflict and computational efficiency and application scenarios of those algorithms are pointed out. In addition, the optimal scheme combining with distributed storage file system is demonstrated. Experimental results show that Davies-Meyer algorithm has a good uniform distribution and high computational efficiency which is suitable for distributed storage systems.

Key words: distributed storage system, hash algorithm, hash distribution, hash conflict, Davies-Meyer algorithm

摘要: 针对分布式存储系统中如何实现数据在物理存储上的均匀分布和高效定位的问题,对多种哈希算法展开研究,提出了衡量分布式存储系统哈希算法优劣的标准;从散列分布性、哈希冲突和计算效率等多个维度对这些哈希算法进行分析比较,指出各种哈希算法的应用场景;结合分布式存储系统的应用,给出最优的哈希算法选择。实验结果证明,Davies-Meyer算法具有很好的均匀分布性和很高的计算效率,很适合分布式存储系统的应用。

关键词: 分布式存储系统, 哈希算法, 散列分布性, 哈希冲突, Davies-Meyer算法