关键词: 任务聚类, 科学工作流, 系统开销, 容错算法, 平衡聚类

Abstract:

In the process of scientific workflow execution, a cluster job composed of multiple tasks has a higher risk of failure than a single task. The fault-tolerant clustering algorithm is faced with load imbalance problems during fault recovery. A Balanced Re-clustering（BR） algorithm is proposed for this purpose. This algorithm combines Horizontal Runtime Balancing（HRB） and Selective Re-clustering（SR） to assign the longest running task to the shortest running class, after re-running the failed task. The experimental results show that compared with the two existing task re-clustering methods, the performance gain of the BR algorithm is up to 84% and 18.75%, respectively, which significantly reduces the workflow execution cost and improves the system’s operating efficiency.

Key words: task clustering, scientific workflow, system overhead, fault tolerance algorithm, balance clustering