关键词: 行人检测, 多模态, 特征金字塔, 特征融合

Abstract: To solve the problems of poor pedestrian detection performance in a single modal due to poor lighting conditions, partial target occlusion and multi-scale target, this paper proposes a multimodal pedestrian detection algorithm based on the fusion of visible and infrared feature pyramids. It uses the deep convolutional neural networks to replace the traditional manual design features, and automatically extracts the features from visible and infrared images. According to the periodic feature maps of ResNet（Residual Net）, a feature pyramid network is built to generate the feature pyramid of each mode. The feature pyramids of each modal are fused layer by layer to create the fusion feature pyramid. It chooses the faster R-CNN algorithm do the following target location and classification algorithm to solve the multispectral pedestrian detection. In addition, in order to solve the problem of ignoring weak features and not effectively integrating complementary features in concatenation fusion and max fusion, the paper proposes a new feature pyramid fusion method. It highlights the strong features and complements the weak features by threshold, effectively utilizes the features of each mode. The multimodal pedestrian detection algorithm based on the fusion of visible and infrared feature pyramids can effectively solve the multimodal pedestrian detection problem, and outperforms state-of-art multimodal pedestrian detectors on the KAIST dataset benchmark.

Key words: pedestrian detection, multimodal, feature pyramid, feature fusion