High Energy Physics Data Placement Strategy Based on Random Forest

doi:10.3778/j.issn.1002-8331.1912-0278

Computer Engineering and Applications ›› 2020, Vol. 56 ›› Issue (21): 60-64.DOI: 10.3778/j.issn.1002-8331.1912-0278

Previous Articles Next Articles

High Energy Physics Data Placement Strategy Based on Random Forest

CHENG Zhenjing, CHENG Yaodong, CHEN Gang, WANG Lu, LI Haibo, HU Qingbao

1.Computing Center, Institute of High Energy Physics, Chinese Academy of Sciences, Beijing 100049, China
2.University of Chinese Academy of Sciences, Beijing 100049, China
3.Tianfu Cosmic Ray Research Center, Institute of High Energy Physics, Chinese Academy of Sciences, Chengdu 610041, China

Online:2020-11-01 Published:2020-11-03

基于随机森林的高能物理数据放置策略

程振京，程耀东，陈刚，汪璐，李海波，胡庆宝

1.中国科学院高能物理研究所，北京100049
2.中国科学院大学，北京100049
3.中国科学院高能物理研究所天府宇宙线研究中心，成都 610041

Abstract

Abstract:

With the continuous developments of high energy physics experiments such as Large High Air Altitude Shower Observatory（LHAASO）, a large amount of data at PB scale will be collected, stored and analyzed every year. At present, random data placement strategy which doesn’t fully consider the differences among data access scenarios, servers and storage devices is generally used. A data placement strategy based on random-forest algorithm is proposed. Storage devices are separated into storage pools（Fast pool, Normal pool） according to their performance. The algorithm will predict and identify a new file’s access pattern, and find one best place for it considering the load of target devices. This paper evaluates the performance of the algorithm with data samples collected from production storage system of LHAASO experiment.

Key words: random forest, distributed storage system, heterogeneous storage, storage pool, data placement strategy, access scenario

摘要：

随着LHAASO高海拔宇宙线等高能物理实验规模的不断扩大，每年需要存储PB级的海量物理数据。高能物理海量存储系统一般采用随机的数据放置策略，没有考虑数据访问场景和服务器节点、存储设备的差异性。针对以上问题，提出一种异构存储环境下基于随机森林算法的数据放置策略，根据存储设备性能差异划分快慢存储池，同时对后期文件的读写访问场景进行预测和识别，综合考虑当前设备负载为数据找到最佳的放置位置。使用真实物理实验数据验证了算法的有效性。

关键词: 随机森林, 分布式存储系统, 异构存储, 存储池, 数据放置策略, 访问场景

CHENG Zhenjing, CHENG Yaodong, CHEN Gang, WANG Lu, LI Haibo, HU Qingbao. High Energy Physics Data Placement Strategy Based on Random Forest[J]. Computer Engineering and Applications, 2020, 56(21): 60-64.

程振京，程耀东，陈刚，汪璐，李海波，胡庆宝. 基于随机森林的高能物理数据放置策略[J]. 计算机工程与应用, 2020, 56(21): 60-64.

[1]	YANG Yemin, ZHANG Huijun, ZHANG Xiaolong. Research on Interpretable Visual Analysis Method of Random Forest [J]. Computer Engineering and Applications, 2021, 57(6): 168-175.
[2]	XIONG Jian, QIN Renchao, HE Mengyi, LIU Jianlan, TANG Fengyang. Application of Improved Random Forest Algorithm in Android Malware Detection [J]. Computer Engineering and Applications, 2021, 57(3): 130-136.
[3]	AN Lei, HAN Zhonghua, LIN Shuo, SHANG Wenli. Research on GAN-SDAE-RF Model for Network Intrusion Detection [J]. Computer Engineering and Applications, 2021, 57(21): 155-164.
[4]	WU Weijie, ZHANG Jingxiang. Random Forest Feature Selection Algorithm Based on Categorization Information and Application [J]. Computer Engineering and Applications, 2021, 57(17): 147-156.
[5]	YAN Zhengxu, QIN Chao, SONG Gang. Random Forest Model Stock Price Prediction Based on Pearson Feature Selection [J]. Computer Engineering and Applications, 2021, 57(15): 286-296.
[6]	ZHU Di, CHEN Danwei. Technology of Mobile Application Identification Based on Density-Based Clustering and Random Forest [J]. Computer Engineering and Applications, 2020, 56(4): 63-68.
[7]	HU Qingyu, LIU Guangchen. Application of Deep Belief Network in Recognition of Protein Coding Regions [J]. Computer Engineering and Applications, 2020, 56(4): 247-255.
[8]	LI Ling, GU Xiaomei, LIU Zihao. Application Research of Multi-subdomain Random Forest in Context-Aware Recommendation [J]. Computer Engineering and Applications, 2020, 56(22): 132-141.
[9]	LUO Jigen, DU Jianqiang, NIE Bin, LI Huan, NIE Jianhua, CHEN Yufeng. Random Forest Optimization Method Based on Cluster Undersampling Strategy [J]. Computer Engineering and Applications, 2020, 56(22): 166-172.
[10]	LI Jieqi, HU Liangbing. Review of Machine Learning for Predictive Maintenance [J]. Computer Engineering and Applications, 2020, 56(21): 11-19.
[11]	WU Shili, TANG Zhenmin, LIU Yong. Fatigue Driving Recognition Algorithm Using Random Forest with Multi-feature Fusion [J]. Computer Engineering and Applications, 2020, 56(20): 212-219.
[12]	GOU Jijun, LI Junhua, CHEN Chen, CHEN Yiming, LV Yida. Network Intrusion Detection Method Based on Random Forest [J]. Computer Engineering and Applications, 2020, 56(2): 82-88.
[13]	SUN Fuxiong, LIU Guangming, ZENG Zixuan, PENG Mengqi. Research on Stock Suspension Prediction Based on Combination Model [J]. Computer Engineering and Applications, 2020, 56(18): 272-278.
[14]	GUO Yuhan, HU Dejia. Vehicle Multiplication Solution Based on Random Forest and Variable Neighborhood Decline [J]. Computer Engineering and Applications, 2020, 56(13): 243-253.
[15]	ZHANG Jiawei, GUO Linming, YANG Xiaomei. Improved Oversampling and Random Forest Algorithm for Imbalanced Data [J]. Computer Engineering and Applications, 2020, 56(11): 39-45.

High Energy Physics Data Placement Strategy Based on Random Forest

基于随机森林的高能物理数据放置策略

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics