Multi-scale changes of pedestrians in many scenes seriously affect the accuracy of the detector, therefore, two modules of convolution feature reconstruction and channel attention are designed to enhance the detection effect of multi-scale pedestrians. Feature pyramids are reconstructed based on the multi-scale features of the original input and feature fusion. Then, the same scale features in multiple feature pyramids are fused to learn the channel attention of each feature layer, and the effective channel layer weight is increased by the weight, so that the features obtained can be used for the final detection. The two modules are integrated into the RFBnet model, and the model loss function is improved to optimize the detection effect of occluded pedestrians. The test results of Caltech-USA, INRIA and ETH data sets show that the accuracy of the new method is higher than that of some multi-scale methods such as RFBnet and MS-CNN, achieving the optimal detection effect on the test subsets of multi-scale pedestrians.