关键词: Tensorflow, CPU+GPU, 卷积神经网络, Swish激活函数, ReLU激活函数, ReLU-Swish激活函数

Abstract: Aiming at the deficiency of the convolutional neural network in the ratio of sex consumption, a collaborative computing model of heterogeneous CPU+GPU is proposed. In the process of model calculation, the CPU is responsible for the logical processing and serial computing, so that the GPU executes highly threaded parallel processing tasks. Through experimental tests compared with single GPU training and single CPU training, the experimental results show that the heterogeneous CPU+GPU computing model is more excellent in the performance ratio.Moreover, for the Swish activation function in the convolutional neural network, the calculation of the error gradient caused by the back propagation of the error gradient is large, the convergence rate is slow, and the ReLU activation function has zero derivative in the x negative interval. The resulting negative gradient is set to zero and neurons may not be activated, and a new activation function ReLU-Swish is proposed. Through the test training comparison and analysis results, the Swish activation function is less than zero and the ReLU activation function is greater than zero to form a piecewise function, and the test comparison experiment is carried out through three data sets CIFAR-10 and MNIST. The experimental results show that the ReLU-Swish activation function has a significant improvement in the convergence speed and the accuracy of the model test training compared with the Swish activation function and the ReLU activation function.

Key words: Tensorflow, CPU+GPU, convolutional neural network, Swish activation function, ReLU activation function, ReLU-Swish activation function