Multi-Modal Pedestrian Recognition on Double-Terminal Shared Network

doi:10.3778/j.issn.1002-8331.2012-0395

Abstract

Abstract: In order to solve the problem of intra-class difference and modal difference in multi-modal pedestrian recognition, a multi-modal pedestrian recognition method using double-terminal shared network is proposed. Firstly, data processing is carried out for images with different modes by cropping and filling. Then, non-local attention blocks are embedded in the last four convolutional layers of Resnet50, and the improved Resnet50 is used as the backbone network to extract features of pictures with different modes respectively, and then different features are input into the sharing network. Finally, the model is trained by using the clustering loss based on in-class distance and modal difference. Experimental results show that the model with non-local block and clustering loss is more accurate, and the model is more robust.

Key words: multimodal person re-identification, convolutional neural network, cluster loss

摘要： 针对多模态行人重识别中存在较大的类内差异和模态差异的问题，提出了一种使用双端共享网络的多模态行人重识别方法。通过裁剪和填充对不同模态的图片进行数据处理；将Resnet50的后4个卷积层中嵌入非局部注意力块，使用改进的Resnet50作为骨干网络分别对不同模态的图片进行特征提取，再将不同的特征输入共享网络；最后使用基于类内距离和模态差异的聚类损失对模型进行训练。实验结果表明，使用非局部注意力块和聚类损失的模型准确率有所提升，且模型更具有鲁棒性。

关键词: 多模态行人重识别, 卷积神经网络, 聚类损失

LUO Qi, JIAO Minghai. Multi-Modal Pedestrian Recognition on Double-Terminal Shared Network[J]. Computer Engineering and Applications, 2022, 58(13): 235-240.

罗琪, 焦明海. 双端可共享网络的多模态行人重识别方法[J]. 计算机工程与应用, 2022, 58(13): 235-240.

References

[1] ZHENG Z，ZHENG L，YANG Y.Unlabeled sample generated by gan improve the person re-identification baseline in vitro[J].arXiv：1701.07717，2017.
[2] PANDA R，BHUIYAN A，MURINO V，et al.Unsupervised adaptive re-identification in open world dynamic camera networks[J].arXiv：1706.03112，2017.
[3] YE M，SHEN J，LIN G，et al.Deep learning for person re-identification：A survey and outlook[J].arXiv：2001.
04193，2020.
[4] ZHANG Z，JIANG S，HUANG C，et al.RGB-IR cross-modality person ReID based on teacher-student GAN model[J].arXiv：2007.07452，2020.
[5] WU A，ZHENG W S，YU H X，et al.RGB-infrared cross-modality person re-identification[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：5380-5389.
[6] DAI P，JI R，WANG H，et al.Cross-modality person re-identification with generative adversarial training[C]//Proceedings of International Joint Conference on Artifical Intelligence，2018：1-2.
[7] WANG G，ZHANG T，CHENG J，et al.RGB-infrared cross-modality person re-identification via joint pixel and feature alignment[C]//Proceedings of the IEEE International Conference on Computer Vision，2019：3623-3632.
[8] WANG Z，WANG Z，ZHENG Y，et al.Learning to reduce dual-level discrepancy for infrared-visible person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2019：618-626.
[9] LIU H，CHENG J.Enhancing the discriminative feature learning for visible-thermal cross-modality person re-identification[J].arXiv：1907.09659，2019.
[10] HAO Y，WANG N，LI J，et al.HSME：Hypersphere manifold embedding for visible thermal person re-identification[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：8385-8392.
[11] YE M，LAN X，LENG Q，et al.Cross-modality person re-identification via modality-aware collaborative ensemble learning[J].IEEE Transactions on Image Processing，2020，29：9387-9399.
[12] YE M，WANG Z，LAN X，et al.Visible thermal person re-identification via dual-constrained top-ranking[C]//Proceedings of International Joint Conference on Artifical Intelligence，2018，1：2.
[13] YE M，LAN X，WANG Z，et al.Bi-directional center-constrained top-ranking for visible thermal person re-identification[J].IEEE Transactions on Information Forensics and Security，2019，15：407-419.
[14] WANG X，GIRSHICK R，GUPTA A，et al.Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7794-7803.
[15] ALEX D，SAMI Z，BANERJEE S，et al.Cluster loss for person re-identification[C]//Proceedings of the 11th Indian Conference on Computer Vision，Graphics and Image Processing，2018：1-8.
[16] NGUYEN D T，HONG H G，KIM K W，et al.Person recognition system based on a combination of body images from visible light and thermal cameras[J].Sensors，2017，17（3）：605.
[17] YE M，WANG Z，LAN X，et al.Visible thermal person re-identification via dual-constrained top-ranking[C]//Proceedings of International Joint Conference on Artifical Intelligence，2018.
[18] FENG Z，LAI J，XIE X.Learning modality-specific representations for visible-infrared person re-identification[J].IEEE Transactions on Image Processing（TIP），2020，29：579-590.
[19] HAO Y，WANG N，GAO X，et al.Dual-alignment feature embedding for cross-modality person re-identification[C]//Proceedings of ACM Multimedia（ACM MM），2019：57-65.