关键词: AlexNet, 卷积神经网络（CNN）, Normalization, 优化器, 激活函数

Abstract:

In this paper, the AlexNet convolutional neural network is improved and optimized by three aspects：Normali-
zation, optimizer and activation function. Firstly, there is no learnable parameter for LRN（Local Response Normalization）. It is proposed to use WN（Weight Normalization） instead of LRN, and put WN after all the pooling layer. The accuracy of the training of AlexNet model is improved. Secondly, the effects of Adam, RMSProp and Momentum on the training of AlexNet model under different learning rates are compared and analyzed, and the corresponding optimization interval of learning rate is obtained. It improves the accuracy of AlexNet’s learning rate interval selection in Optimizer. Finally, for the partial weight of the ReLU activation function in AlexNet can not be updated and the gradient explosion problem, it proposes a fusion segmentation function algorithm of ReLU6 and Swish, which improves the AlexNet model training convergence speed and accuracy while also alleviating the occurrence of overfitting.

Key words: AlexNet, Convolutional Neural Network（CNN）, Normalization, optimizer, activation function