Abstract: Since online abnormal detection for backbone network with large flow currently is a research hotspot in network security field, an online network abnormal detection method is proposed to handle big data stream properly. The method processes big data stream into micro-clusters with density-based cluster method, and then micro-clusters absorb data stream directly to enhance the performance. The method regularly executes outlier detection process to find intrusion. The method does not require offline training process and can find any arbitrary clusters. It also supports big data stream and can balance between detection precision and resources with great performance. In the experiment, the prototype system finishes analysis task in 20 s over MIT Lincoln Laboratory LLS_DDOS_1.0 data, with 82% TPR and 6% FPR, which is equivalent to K-means.

Key words: density based clustering, micro-cluster, data stream, outlier detection

摘要： 针对大流量骨干网的在线网络异常检测是目前网络安全研究的热点之一，提出一种网络异常检测方法，有效在线处理大数据流，利用密度聚类算法把大数据流转换成微簇，通过微簇提高处理效率，定时调用孤立点检测算法发现攻击行为。方法具有不需线下训练、能发现任意行为模式、支持大数据流、可以平衡检测精度与系统资源要求、处理效率高等优点。实验表明，原型系统在20 s完成2000年LLS_DDOS_1.0数据集分析，检测率为82%，误报率为6%，效果与K-means相当。

关键词: 密度聚类, 微簇, 数据流, 孤立点检测