关键词: PE文件, 恶意软件检测, 深度学习, 神经网络, 深度神经网络（DNN）

Abstract:

Most of the current deep-learning-based malware detection methods have the problem of poor generalization caused by the model structures and sample preprocessing methods that are not suitable enough. In other words, the trained malware detection models might have a poor detection effect on those malwares that are not included in the training sample set or those newly emerged malwares. This paper proposes an improved Deep Neural Network（DNN） based malware detection method, which uses multiple fully connected layers to build a malware detection model, and introduces a directional Dropout regularization method to prune the weights in the neural network during the model training process. The experimental results on the Virusshare dataset and the lynx-project sample set show that, compared with another DNN based malware detection model DeepMalNet, the proposed model attains an average predicted probability on the malicious PE sample set that is increased by 0.048, and an average predicted probability on the packed normal sample set that is decreases by 0.64. The results indicate that the proposed method has a better generalization ability, and a better detection effect on malwares outside the training sample set.

Key words: PE file, malware detection, deep learning, neural network, Deep Neural Network（DNN）