Computer Engineering and Applications ›› 2012, Vol. 48 ›› Issue (17): 88-92.

Previous Articles     Next Articles

Performance optimization of fine-grained I/O in parallel file system Lustre

LI Linlin, WU Wanguo, SUN Liangxu   

  1. School of Software, Liaoning University of Science & Technology, Anshan, Liaoning 114045, China
  • Online:2012-06-11 Published:2012-06-20

并行文件系统Lustre细粒度I/O性能优化

李林林,吴万国,孙良旭   

  1. 辽宁科技大学 软件学院,辽宁 鞍山 114045

Abstract: Performance of coarse-grained I/O is good in parallel file system Lustre, performance of fine-grained I/O is relatively low compared with coarse-grained I/O, so performance optimization of fine-grained I/O is the key problem to improve I/O performance of overall system. On the research and analysis of Lustre I/O mode, fine-grained I/O service process and page replacement algorithm etc, it puts forward fine-grained priority FGF-LRU algorithm. It maximizes to retain fine-grained I/O pages in page cache of OST terminal and client terminal, reduces the page sinking speed caused by fine-grained I/O, extends time when fine-grained I/O page is in memory, thereby reduces the times of disk accesses and reduces disk access cost. It verifies the effectiveness of FGF-LRU algorithm based on comparison and analysis of the test data. It improves performance of fine-grained I/O without affecting performance of coarse-grained I/O, and ultimately improves I/O performance of the overall system.

Key words: parallel file system, Lustre, fine-grained, Input/Output(I/O)

摘要: 并行文件系统Lustre粗粒度I/O性能良好,细粒度I/O性能相对粗粒度I/O比较低下,因此优化细粒度I/O性能成为提高系统整体I/O性能的关键问题。在研究和分析了Lustre的I/O访问模式、细粒度I/O服务流程和页面替换算法等方面后,提出了细粒度优先(Fine Grained First,FGF)LRU算法。在OST端及Client端的页高速缓存中最大程度地保留细粒度I/O的页面,降低细粒度I/O引起的页面下沉速度,延长细粒度I/O页面在主存中的时间,进而减少对磁盘的访问次数,降低磁盘访问开销。通过对实验数据的对比和分析,验证了FGF-LRU算法的有效性。在不影响粗粒度I/O性能的情况下,提高了细粒度I/O性能,最终实现提高系统整体I/O性能。

关键词: 并行文件系统, Lustre, 细粒度, 输入/输出(I/O)