Elephant Flow Detection Method Based on Adaptive Threshold

doi:10.3778/j.issn.1002-8331.2008-0206

Abstract

Abstract: Aiming at the difficulty in detecting abnormal traffic in data center networks, an adaptive threshold elephant flow detection system is proposed. The system combines the flexibility and global visibility of data center network. The system uses the weighted optimized dynamic traffic learning method based on Gaussian distribution to predict the detection threshold of elephant flow in real time. It can reduce the detection error rate. The detection threshold update frequency is reduced through a smoothing mechanism based on difference estimation. Experimental results show that the system can effectively identify large and small flow in the data center network, and the recognition error rate is low. The smoothing mechanism reduces the flow table jitter, the control plane overhead and detection delay are relatively low, and real-time effective monitoring of data center network traffic is realized.

Key words: data center, elephant flow detection, dynamic flow learning, smoothing mechanism

摘要： 针对数据中心网络异常流量检测难的问题，提出一种自适应阈值的大象流检测系统。系统结合数据中心网络高度灵活性和全局可见性的特点，采用基于高斯分布的加权优化动态流量学习方法实时预测大象流检测阈值，降低检测错误率，通过基于差分估计的平滑机制，降低检测阈值配置更新频率。仿真实验结果表明，该系统可以有效识别数据中心网络中的大小流，识别错误率较低，通过平滑机制处理减少了流表抖动，控制平面的开销和检测时延相对较低，实现了数据中心网络流量的实时有效监控。

关键词: 数据中心, 大象流检测, 动态流量学习, 平滑机制

LIU Yi, LI Jianhua, CHEN Yu. Elephant Flow Detection Method Based on Adaptive Threshold[J]. Computer Engineering and Applications, 2022, 58(3): 159-164.

刘奕, 李建华, 陈玉. 基于自适应阈值的大象流检测方法[J]. 计算机工程与应用, 2022, 58(3): 159-164.

References

[1] 魏祥麟，陈鸣，范建华，等.数据中心网络的体系结构[J].软件学报，2013，24（2）：295-316.
WEI X L，CHEN M，FAN J H，et al.Architecture of the data center network[J].Journal of Software，2013，24（2）：295-316.
[2] BENSON T，AKELLA A，MALTZ D A.Network traffic characteristics of data centers in the wild[C]//Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement，2010.
[3] KANDULA S，SENGUPTA S，GREENBERG A.et al.The nature of data center traffic：measurements & analysis[C]//Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement，2009.
[4] GREENBERG A，HAMILTON J R，JAIN N，et al.VL2：a scalable and flexible data center network[J].Communications of the ACM，2009，54（3）：95-104.
[5] 张朝昆，崔勇，唐翯祎，等.软件定义网络（SDN）研究进展[J].软件学报，2015，26（1）：62-81.
ZHANG C K，CUI Y，TANG G W，et al.State-of-the-art survey on software-defined networking[J].Journal of Software，2015，26（1）：62-81.
[6] CURTIS A R，KIM W，YALAGANDULA P.Mahout：low-overhead datacenter traffic management using end-host-based elephant detection[C]//INFOCOM，2011 Proceedings IEEE，2011：1629-1637.
[7] LIN C Y，CHEN C，CHANG J W，et al.Elephant flow detection in datacenters using openflow-based hierarchical statistics pulling[C]//Global Communications Conference （GLOBECOM），2014：2264-2269.
[8] LEE E K，VISWANATHAN H，POMPILI D.Model-based thermal anomaly detection in cloud datacenters using thermal imaging[J].IEEE Transactions on Cloud Computing，2018，6：330-343.
[9] KIRKPATRICK K.Software-defined networking[J].Communications of the ACM，2013，56（9）：16-19.
[10] AL-FARES M，RADHAKRISHNAN S，RAGHAVAN B，et al.Hedera：dynamic flow scheduling for data center networks[C]//IEEE Inforcom Jose，CA，USA，April 28-30，2010：1-19.
[11] DUFFIELD N，LUND C，THORUP M.Sample less：control of volume and variance in network measurement[J].IEEE Transactions on Information Theory，2005，51（5）：1756-1775.
[12] 裴育杰，王洪波，程时端.基于两级LRU机制的大流检测方法[J].电子学报，2009，37（4）：684-691.
PEI Y J，WANG H B，CHENG S R，et al.A dual-LRU based algorithm for identifying and measuring large flows[J].Acta Electronica Sinica，2009，37（4）：684-691.
[13] 孙昱，夏靖波，赵小欢，等.基于LEAST和CBF两级结构的大流检测方法[J].华中科技大学学报（自然科学版），2014，42（4）：40-44.
SUN Y，XIA J B，ZHAO X H，et al.LEAST and CBF two-level architecture based algorithm for identifying and measuring large flows[J].Journal of Huazhong University of Science and Technology（Natural Science Edition），2014，42（4）：40-44.
[14] ZHANG Z，WANG B，LAN J.Identifying elephant flows in internet backbone traffic with bloom filters and LRU[J].Computer Communications，2015，61（1）：70-78.
[15] BI C，LUO X，YE T.et al.On precision and scalability of elephant flow detection in data center with SDN[C]//IEEE Globecom Workshops，2013.