In order to solve the problem of low accuracy of the existing Convolutional Neural Network（CNN） in gesture recognition, a compound gesture recognition method based on feature fusion of dual-channel CNN and dynamic attenuation learning rate is proposed. The features of gesture images can be extracted by two independent channels. Firstly, the first channel composed of SENet（Squeeze-and-Excitation Networks） is used to extract global features. Secondly, local features are extracted by the second channel composed of RBNet （Residual Block Networks）. Then, the global features and local features are merged into the channel dimension, so that the network can learn more comprehensive gesture feature information. Meanwhile, the learning rate of dynamic attenuation is used to train the dual-channel network model, for improving the convergence speed and stability of the model. Compared with the experimental results of other CNN models, the proposed compound gesture recognition method has higher gesture recognition rate, fewer parameters, and is suitable for the recognition of different gesture image data sets.