融合流能量模型的流量异常检测方法

doi:10.3778/j.issn.1002-8331.2307-0307

摘要/Abstract

摘要： 网络流量异常检测是一种可以协助识别和预防恶意网络攻击的关键网络安全技术。现有的网络流量异常检测方法通常依赖于复杂的机器学习模型和大量的标记数据，因此，这些方法在不重新训练模型的情况下难以应用于不同场景，无法实时有效地处理大规模的、持续发生的网络攻击。针对这些问题，提出了一种基于网络流能量模型的分类方法，使用逆统计物理学模型学习网络中目标流量特征，能够以宏观的现实观测或现实数据为基础，不再依赖人工标注。结合能量模型的概念构建网络流量识别模型，通过该模型判断样本是否与主体统计分部相符。具体来讲，通过能量模型中的局部场和耦合场分别描述流量包之间的个体行为特征以及相互行为特征。结合以上两种特征来计算样本的能量，若能量小于或等于阈值，则样本与主体分布相符，说明该样本为正常数据，否则判断为异常数据。由于该方法不依赖于人工标注，能够适应多种网络环境，且无须重复训练，进而能够解决当前流量异常检测方法无法适应不同场景，以及需要大量标注等问题。为了评估该方法的有效性，使用数据集Kitsune-2018和CTU-13对方法进行验证。实验结果表明，该方法在网络流量分类任务中能够取得较好的分类效果和性能表现，进一步说明该方法能够准确地执行网络流分类任务并且能够适应场景变化。

关键词: 流量分类, 网络流量, 流分类器, 能量模型, 逆统计物理学模型

Abstract: Abnormal network traffic detection is a key cybersecurity technology that assists in identifying and preventing malicious network attacks. Existing methods for detecting abnormal network traffic typically rely on complex machine learning models and a large amount of labeled data. Consequently, these methods are challenging to apply to different scenarios without retraining the model and cannot effectively handle large-scale, ongoing network attacks in real-time. To address these issues, this paper proposes a classification method based on a network flow energy model. It utilizes a reverse statistical physics model to learn target traffic features in the network, allowing it to be based on macroscopic real observations or real data without the need for manual labeling. Subsequently, the paper combines the concept of the energy model to construct a network traffic recognition model. This model judges whether a sample conforms to the main statistical distribution. Specifically, the method describes individual behavior characteristics and interaction features between traffic packets through the local field and coupling field in the energy model. By combining these two features, the method calculates the sample’s energy. If the energy is below a threshold, the sample aligns with the main distribution, indicating normal data; otherwise, it is considered abnormal data. As this method does not rely on manual labeling, it can adapt to various network environments without the need for repetitive training. This addresses current issues in traffic abnormality detection methods, which struggle to adapt to different scenarios and require extensive labeling. To evaluate the effectiveness of this method, the paper validates it using the Kitsune-2018 and CTU-13 datasets. Experimental results demonstrate that the proposed method achieves good classification performance and overall effectiveness in network traffic classification tasks. This further indicates its accuracy in performing network flow classification tasks and its adaptability to changing scenarios.

Key words: network flow classification, network flow, flow-based classifier, energy model, reverse statistical physics model

杜文勇, 徐李阳, 王晨飞, 赵文华, 张烁, 谢瑞楠, 曹彭程, 李晓红. 融合流能量模型的流量异常检测方法[J]. 计算机工程与应用, 2024, 60(20): 274-283.

DU Wenyong, XU Liyang, WANG Chenfei, ZHAO Wenhua, ZHANG Shuo, XIE Ruinan, CAO Pengcheng, LI Xiaohong. Network Traffic Classification Method Fused with Flow Energy Model[J]. Computer Engineering and Applications, 2024, 60(20): 274-283.

参考文献

[1] THAKKAR A, LOHIYA R. A survey on intrusion detection system: feature selection, model, performance measures, application perspective, challenges, and future research directions[J]. Artificial Intelligence Review, 2022, 55(1): 453-563.
[2] MAURYA S, KUMAR S, GARG U, et al. An efficient framework for detection and classification of IoT botnet traffic[J]. ECS Sensors Plus, 2022, 1(2): 026401.
[3] KUNDU P P, TRUONG-HUU T, CHEN L, et al. Detection and classification of botnet traffic using deep learning with model explanation[J]. IEEE Transactions on Dependable and Secure Computing, 2022: 3183361.
[4] BISWAS R, ROY S. Botnet traffic identification using neural networks[J]. Multimedia Tools and Applications, 2021, 80: 24147-24171.
[5] APRUZZESE G, ANDREOLINI M, FERRETTI L, et al. Modeling realistic adversarial attacks against network intrusion detection systems[J]. Digital Threats: Research and Practice, 2022, 3(3): 1-19.
[6] RIBEIRO A R L, SANTOS R Y C, NASCIMENTO A C A. Anomaly detection technique for intrusion detection in SDN environment using continuous data stream machine learning algorithms[C]//Proceedings of the 2021 IEEE International Systems Conference, 2021: 1-7.
[7] SOMMER R, PAXSON V. Outside the closed world: on using machine learning for network intrusion detection[C]//Proceedings of the 2010 IEEE Symposium on Security and Privacy, 2010: 305-316.
[8] PONTES C F, SOUZA D M M, GONDIM J J, et al. A new method for flow-based network intrusion detection using the inverse Potts model[J]. IEEE Transactions on Network and Service Management, 2021, 18(2): 1125-1136.
[9] UMER M F, SHER M, BI Y. Flow-based intrusion detection: techniques and challenges[J]. Computers & Security, 2017, 70: 238-254.
[10] VERMA A, RANGA V. Statistical analysis of CIDDS-001 dataset for network intrusion detection systems using distance-based machine learning[J]. Procedia Computer Science, 2018, 125: 709-716.
[11] RING M, LANDES D, HOTHO A, et al. Detection of slow port scans in flow-based network traffic[J]. PLoS One, 2018, 13(9): 0204507.
[12] ABDULHAMMED R, FAEZIPOUR M, ABUZNEID A, et al. Deep and machine learning approaches for anomaly-based intrusion detection of imbalanced network traffic[J]. IEEE Sensors Letters, 2019, 3: 1-4.
[13] SONG S, LING L, MANIKOPOULO C N. Flow-based statistical aggregation schemes for network anomaly detection[C]//Proceedings of the IEEE International Conference on Networking, Sensing and Control, 2006: 786-791.
[14] TRAN Q A, JIANG F, HU J. A real-time netflow-based intrusion detection system with improved BBNN and high-frequency field programmable gate arrays[C]//Proceedings of the 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications, 2012: 201-208.
[15] NGUYEN H A, VAN NGUYEN T, KIM D I, et al. Network traffic anomalies detection and identification with flow monitoring[C]//Proceedings of the 2008 5th IFIP International Conference on Wireless and Optical Communications Networks, 2008: 1-5.
[16] JADIDI Z, MUTHUKKUMARASAMY V, SITHIRASENAN E. Metaheuristic algorithms based flow anomaly detector[C]//Proceedings of the 2013 19th Asia-Pacific Conference on Communications, 2013: 717-722.
[17] WINTER P, HERMANN E, ZEILINGER M. Inductive intrusion detection in flow-based network data using one-class support vector machines[C]//Proceedings of the 2011 4th IFIP International Conference on New Technologies, Mobility and Security, 2011: 1-5.
[18] WAGNER C, FRAN?OIS J, STATE R, et al. Machine learning approach for IP-flow record anomaly detection[C]//Proceedings of the 10th International IFIP TC 6 Networking Conference, 2011: 28-39.
[19] SHUBAIR A, RAMADASS S, ALTYEB A A. kENFIS: kNN-based evolving neuro-fuzzy inference system for computer worms detection[J]. Journal of Intelligent & Fuzzy Systems, 2014, 26(4): 1893-1908.
[20] COSTA K A P, PEREIRA L A M, NAKAMURA R Y M, et al. A nature-inspired approach to speed up optimum-path forest clustering and its application to intrusion detection in computer networks[J]. Information Sciences, 2015, 294: 95-108.
[21] HOSSEINPOUR F, AMOLI P V, FARAHNAKIAN F, et al. Artificial immune system based intrusion detection: innate immunity using an unsupervised learning approach[J]. International Journal of Digital Content Technology and Its Applications, 2014, 8(5): 1-12.
[22] ZHAO D, TRAORE I, SAYED B, et al. Botnet detection based on traffic behavior analysis and flow intervals[J]. Computers & Security, 2013, 39: 2-16.
[23] STEVANOVIC M, PEDERSEN J M. An efficient flow-based botnet detection using supervised machine learning[C]//Proceedings of the 2014 International Conference on Computing, Networking and Communications, 2014: 797-801.
[24] HADDADI F, RUNKEL D, ZINCIR-HEYWOOD A N, et al. On botnet behaviour analysis using GP and C4.5[C]//Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, 2014: 1253-1260.
[25] 皇甫雨婷, 李丽颖, 王海洲, 等. 自注意力的多特征网络流量异常检测与分类[J]. 华东师范大学学报 (自然科学版), 2021(6): 161-173.
HUANGFU Y T, LI L Y, WANG H Z, et al. Enabling self-attention based multi-feature anomaly detection and classification of network traffic[J]. Journal of East China Normal University (Natural Science), 2021(6): 161-173.
[26] CAO Y L, JI R W, HUANG X, et al. Empirical mode decomposition-empowered network traffic anomaly detection for secure multipath TCP communications[J]. Mobile Networks and Applications, 2022, 27(6): 2254-2263.
[27] BAKHSHI T, GHITA, B. Anomaly detection in encrypted internet traffic using hybrid deep learning[J]. Security and Communication Networks, 2021(1): 1-16.
[28] WAWROWSKI L, BIALAS A, KAJZER A, et al. Anomaly detection module for network traffic monitoring in public institutions[J]. Sensors, 2023, 23(6): 2974.
[29] CARRERA F, DENTAMARO V, GALANTUCCI S, et al. Combining unsupervised approaches for near real-time network traffic anomaly detection[J]. Applied Sciences, 2022, 12(3): 1759.
[30] DUTTA V, PAWLICKI M, KOZIK R, et al. Unsupervised network traffic anomaly detection with deep autoencoders[J]. Logic Journal of the IGPL, 2022, 30(6): 912-925.
[31] ZHU S, XU X, GAO H, et al. CMTSNN: a deep learning model for multi-classification of abnormal and encrypted traffic of Internet of things[J]. IEEE Internet of Things Journal, 2023, 10(13): 11773-11791.
[32] KUROSE, J, KEITH W. Computer networking: a top-down approach edition[M]. Massachusetts: Addision Wesley, 2007.
[33] JAYNES, EDWIN T. Information theory and statistical mechanics[J]. Physical Review, 1957, 106(4): 620.
[34] LESTARI W, SUMARLINDA S. Implementation of K-nearest neighbor (KNN) and suport vector machine (SVM) for clasification cardiovascular disease[J]. International Journal of Multi Science, 2022, 2(10): 30-36.
[35] ZHANG F, SHANG T, LIU J. Imbalanced encrypted traffic classification scheme using random forest[C]//Proceedings of the 2020 International Conferences on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) and IEEE Congress on Cybermatics (Cybermatics), 2020: 837-842.
[36] QI H, WANG J, LI W, et al. A blockchain-driven IIoT traffic classification service for edge computing[J]. IEEE Internet of Things Journal, 2020, 8(4): 2124-2134.
[37] SMADIA S, ALMOMANIB O, MOHAMMADC A, et al. VPN encrypted traffic classification using XGBoost[J]. International Journal, 2021: 24066625.