Computer Engineering and Applications ›› 2020, Vol. 56 ›› Issue (13): 77-83.DOI: 10.3778/j.issn.1002-8331.1906-0169

Previous Articles     Next Articles

Research on Random Access Method of Alluxio

WEI Zhanchen, HUANG Qiulan, SUN Gongxing, LIU Xiaoyu, WANG Yi   

  1. 1.Institute of High Energy Physics, Chinese Academy of Sciences, Beijing 100049, China
    2.University of Chinese Academy of Sciences, Beijing 100049, China
  • Online:2020-07-01 Published:2020-07-02



  1. 1.中国科学院 高能物理研究所,北京 100049
    2.中国科学院大学,北京 100049


Alluxio simplifies the implementation of file system to pursue scalability and fault tolerance, like not supporting data random access, which leads it cannot be available to many applications. Also Alluxio’s native Java interface is not flexible to support traditional programs and is not able to fully use the high-speed performance of memory. In order to address these issues, a new method to support data random access is proposed. The new method allows the tasks to random read/write data on the local file system as well local memory instead of using original Alluxio data streaming interface. With random access to local memory, this work can greatly improve the read/write performance. Test results show that write performance can be improved by 14.5% and read performance can be increased by more than 1.4 times compared to the native Java interface. Combined with application, this new method can achieve up to tens times of performance improvement.

Key words: Alluxio distributed in-memory storage system, data random access, memory computing, memory mapping, scientific computing



关键词: Alluxio分布式内存存储系统, 数据随机访问, 内存计算, 内存映射, 科学计算