关键词: 入侵检测, 深度强化学习, 双重深度Q网络, 卷积神经网络（CNN）, 长短期记忆网络（LSTM）

Abstract: As an effective defense method of network security, intrusion detection technology is an essential part of network security system. With the drastic development of the Internet, the amount of network data increases rapidly, and network attacks tend to be more complex and diversified, consequently, current intrusion detection technologies cannot identify all kinds of attacks effectively. Owing to the unbalanced problem between normal traffic and attack traffic in the real network environment and the low detection rate of attack traffic, this paper proposes a CBL_DDQN detection model based on improved double deep Q-network which is based on deep reinforcement learning. A hybrid model consisting of one-dimensional convolutional neural network and bi-directional long short-term memory network is utilized in the DDQN framework of deep reinforcement learning, then the feedback learning and strategy-generating mechanism of deep reinforcement learning is used for training the agent to classify different types of attack samples, which can greatly weaken the dependence on data labels in the process of training model. In the meantime, the Borderline-SMOTE algorithm is used to reduce data imbalance so as to improve the detection rate of rare attack traffic. The performance of the model evaluated by NSL_KDD and UNSW_NB15 datasets shows that the model performs well in accuracy, precision and recall. The detection result of the model is far better than that of Adam-BNDNN, KNN, SVM and other detection methods, which implies the intrusion detection model proposed in this paper is efficient.

Key words: intrusion detection, deep reinforcement learning, double deep Q-network, convolutional neural network（CNN）, long short-term memory network（LSTM）