Abstract: The Storm cluster provides powerful real-time processing ability. Storm node running different task in different speed results in Tuple timeout, and thus affects the system throughput and performance. This paper proposes a backpressure mechanism that can adjust data load of each section of the Topology in a flexible way. According to the load condition, the mechanism adopts variable queue to adapt to the dynamic change of the load data and improves the system throughput. The experiments show that this mechanism can avoid the shock of data flow and improve the performance and stability of Storm.

Key words: Storm cluster, backpressure, variable queue, data load, throughput

摘要： Storm集群提供了强大的实时处理能力，Storm上下游处理节点由于任务差异而导致数据流元组Tuple处理超时从而影响系统吞吐量及其性能。针对该问题，提出了一种能够灵活调节Topology中各环节数据负载的反压机制，该机制采用可变队列，并根据当前Tuple负载动态调整队列大小，以适应数据负载的动态变化，并提升系统吞吐量。实验结果表明，该反压机制能够避免反压过程中出现数据流的震荡，同时提高系统性能和稳定性。

关键词: Storm集群, 反压机制, 可变队列, 数据负载, 吞吐量