计算机工程与应用 ›› 2018, Vol. 54 ›› Issue (7): 84-88.DOI: 10.3778/j.issn.1002-8331.1702-0114

• 大数据与云计算 • 上一篇    下一篇

Storm集群下一种基于Topology的任务调度策略

蒋  溢1,2,罗宇豪1,朱恒伟1   

  1. 1.重庆邮电大学 计算机科学与技术学院,重庆 400065
    2.重庆市移动互联网数据应用工程技术研究中心,重庆 400065
  • 出版日期:2018-04-01 发布日期:2018-04-16

Research of scheduling based on Topology under Storm clusters

JIANG Yi1,2, LUO Yuhao1, ZHU Hengwei1   

  1. 1.School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
    2.Chongqing Engineering Research Center of Mobile Internet Data Application, Chongqing 400065, China
  • Online:2018-04-01 Published:2018-04-16

摘要: Storm作为开源的分布式实时计算系统在业界得到了广泛应用,针对Storm自带调度策略忽略了Topology组件任务间的逻辑耦合性,从而引起大量tuple传输产生较大网络时延问题,结合进程代数将Topology等效简化为具有明显局部可串行化特征的逻辑进程系统模型,并基于该模型采用最大本地化调度策略。实验结果表明,该策略能有效缩短平均流事件处理时延,且具有较高的稳定性和可靠性。

关键词: Storm, 调度策略, 进程代数, Topology, 最大本地化

Abstract: As an open source distributed real-time computing system, Storm has been widely used in the industry. But due to the problem that tuple transmission has cost a considerable latency on cluster network led by the ignorance that components between Topology may have logical coupling, the default scheduling strategy, an optimized scheduling strategy based on Topology is proposed. The strategy introduces CSP algorithm to equivalently simplify a Topology to a logical process system with clear partial serialization features, and a maximum localization scheduling strategy is implemented based on the logical process system. In addition, the experiments show that the strategy can efficiently reduce the average processing latency of tuples and perform well with stability and reliability.

Key words: Storm, scheduling strategy, communicating sequential processes, Topology, maximum localization