关键词: 随机森林, 分布式存储系统, 异构存储, 存储池, 数据放置策略, 访问场景

Abstract:

With the continuous developments of high energy physics experiments such as Large High Air Altitude Shower Observatory（LHAASO）, a large amount of data at PB scale will be collected, stored and analyzed every year. At present, random data placement strategy which doesn’t fully consider the differences among data access scenarios, servers and storage devices is generally used. A data placement strategy based on random-forest algorithm is proposed. Storage devices are separated into storage pools（Fast pool, Normal pool） according to their performance. The algorithm will predict and identify a new file’s access pattern, and find one best place for it considering the load of target devices. This paper evaluates the performance of the algorithm with data samples collected from production storage system of LHAASO experiment.

Key words: random forest, distributed storage system, heterogeneous storage, storage pool, data placement strategy, access scenario