计算机工程与应用 ›› 2019, Vol. 55 ›› Issue (19): 228-234.DOI: 10.3778/j.issn.1002-8331.1806-0032

• 工程与应用 • 上一篇    下一篇

高能物理分布式数据获取架构的设计与实现

李瑾,顾旻皓,李飞,朱科军   

  1. 1.核探测与核电子学国家重点实验室,北京 100049
    2.中国科学院 高能物理研究所,北京 100049
    3.中国科学院大学,北京 100049
  • 出版日期:2019-10-01 发布日期:2019-09-30

Design and Implementation of Distributed Data Acquisition Architecture in High Energy Physics

LI Jin, GU Minhao, LI Fei, ZHU Kejun   

  1. 1.State Key Laboratory of Particle Detection and Electronics, Beijing 100049, China
    2.Institute of High Energy Physics, Chinese Academy of Sciences, Beijing 100049, China
    3.University of Chinese Academy of Sciences, Beijing 100049, China
  • Online:2019-10-01 Published:2019-09-30

摘要: 当代高能物理实验规模庞大且产生的实验数据量急剧增加,向数据获取软件系统提出更为严格、复杂、高性能的处理要求。借助于高速互联网络和强大的处理单元,采用分布式的数据获取软件的数据收集和处理能力得到显著提升,从而更高效地实现粒子物理实验数据获取目标。这种分布式数据获取处理架构在功能上主要负责对多个数据处理进程的运行控制和管理、软硬件配置、状态监控、信息共享和交互等。相较于基于消息传递的分布式协调服务,以Redis为中心的共享内存技术简化了数据获取系统中多进程间的同步,实现轻量级的CAP最终平衡,还作为中心数据库完成数据获取过程中信息的共享需求,是实现分布式处理框架的一种方便、轻巧、实用的解决方案,在当前规模的高能物理实验计算集群中可以可靠、稳定地开发实现和部署运行。

关键词: 数据获取, 分布式处理框架, 分布式系统, Redis, 高海拔宇宙线观测站(LHAASO)

Abstract: The amount of experimental data generated in the large-scaled contemporary high energy physics experiments have increased dramatically, that places more stringent, complex and high-performance requirements on the data acquisition online processing software system. With the high-speed Internet and powerful process units, the data collection and processing capabilities of distributed online processing software have been significantly improved, making data acquisition targets of particle physics experiments have been achieved more efficiently. The framework of distributed online processing takes the charge of the control and management of multiple processes that running on the cluster, hardware and software configuration, status monitoring, information sharing and interaction in functional. Compared to distributed coordination services based on message delivery, the shared memory technology such as Redis simplifies the coordination among multiple processes in the system, achieves the final balance of lightweight CAP, and stores data acquisition information as a central database, which is a convenient, lightweight and practical solution for realizing the online processing framework. In some medium-scale high energy physics experimental computing cluster, it is reliable and stable for development, implementation, and deployment.

Key words: data acquisition, distributed framework of processing, distributed system, Redis, Large High Altitude Air Shower Observatory(LHAASO)