Cross-Modal Re-Identification Light Weight Network Combined with Data Enhancement

doi:10.3778/j.issn.1002-8331.2212-0100

Abstract

Abstract: Among the existing cross modal re-identification methods, the research on lightweight network is less. Considering the requirement of hardware deployment for lightweight network, a new cross modal re-identification lightweight network is proposed. Based on Osnet ,the feature extractor and feature embedder are split. At the same time, data enhancement operations are used to maximize the use of limited data sets to improve network robustness, and the hard triplet loss is improved to further reduce the computation and reduce the difference between modals, so as to improve the accuracy of network identification. The network is lightweight, simple in structure and remarkable in effect. In the all search mode of SYSU-MM01 dataset, the rank-1/mAP of the proposed method reaches 65.56%,61.36% respectively, and the number of parameters is only 1.92×106.

Key words: depth separable convolution, person re-identification, lightweight network, hard triplet loss function

摘要： 现有的跨模态行人重识别方法中，轻量化网络的相关研究较少。考虑到硬件部署对轻量化网络的需求，提出新的跨模态行人重识别轻量网络。以Osnet为基础，进行特征提取器和特征嵌入器的拆分。同时使用数据增强操作，利用有限的数据集，最大程度提高了网络的鲁棒性。改进难样本三元组损失函数，在减少计算量的同时缩小模态间差异，提升网络识别准确率。提出的轻量化网络结构简单且效果显著，在SYSU-MM01数据集的全搜索模式下rank-1/mAP分别达到65.56%、61.36%，参数量仅为1.92×106。

关键词: 深度可分离卷积, 行人重识别, 轻量化网络, 难样本三元组损失函数

CAO Ganggang, WANG Banghai, SONG Yu. Cross-Modal Re-Identification Light Weight Network Combined with Data Enhancement[J]. Computer Engineering and Applications, 2024, 60(8): 131-139.

曹钢钢, 王帮海, 宋雨. 结合数据增强的跨模态行人重识别轻量网络[J]. 计算机工程与应用, 2024, 60(8): 131-139.

References

[1] YE M, SHEN J B, SHAO L.Visible-infrared person re-identification via homogeneous augmented tri-modal learning[J]. IEEE Transactions on Information Forensics and Security, 2021, 16: 728-739.
[2] ZHU Y X, ZHAO Y, LI W, et al. Hetero-center loss for cross-modality person re-identification[J]. Neurocomputing, 2020, 386: 97-109.
[3] HOWARD A G, ZHU M, CHEN B, et al. MobileNets: efficient convolutional neural networks for mobile vision applications[J]. arXiv:1704.04861, 2017.
[4] CHOLLET F. Xception: deep learning with depthwise separable convolutions[J].arXiv:1610.02357, 2016.
[5] SIFRE L, MALLAT S. Rigid-motion scattering for texture classification[J]. arXiv:1403.1687, 2014.
[6] SANDLER M, HOWARD A, ZHU M, et al. MobileNetV2: inverted residuals and linear bottlenecks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[7] ZHOU K, YANG Y, CAVALLARO A, et al. Learning generalisable omni-scale representations for person re-identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 44: 5056-5069.
[8] SHESHKAL S A, FOULAD-GHALEH K, AGHABABA H.An improved person re-identification method by light-weight convolutional neural network[C]//Proceedings of the 2020 10th International Conference on Computer and Knowledge Engineering, 2020.
[9] LI W, ZHU X, GONG S. Harmonious attention network for person re-identification[J].arXiv:1802.08122, 2018.
[10] LIN M, CHEN Q, YAN S. Network in network[J].arXiv:1312.4400, 2014.
[11] YE M, WANG Z, LAN X Y, et al. Visible thermal person re-identification via dual-constrained top-ranking[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018.
[12] LIU H, TAN X, ZHOU X. Parameter sharing exploration and hetero-center based triplet loss for visible-thermal person re-identification[J].IEEE Transactions on Multimedia, 2021, 23: 4414-4425.
[13] ZHONG Z, ZHENG L, KANG G, et al. Random erasing data augmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2017.
[14] MüLLER S G, HUTTER F.TrivialAugment: tuning-free yet state-of-the-art data augmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
[15] CUBUK E D, ZOPH B, MANE D, et al. AutoAugment: learning augmentation strategies from data[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019.
[16] RADENOVIC F, TOLIAS G, CHUM O. Fine-tuning CNN image retrieval with no human annotation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(7): 1655-1668.
[17] HERMANS A.In defense of the triplet loss for person re-identification[J].arXiv:1703.07737, 2017.
[18] WU A, ZHENG W S, YU H X, et al. RGB-infrared cross-modality person re-identification[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, 2017.
[19] NGUYEN D T, HONG G H, KIM K W, et al. Person recognition system based on a combination of body images from visible light and thermal cameras[J]. Sensors, 2017, 17(3): 605.
[20] YE M, SHEN J, LIN G, et al. Deep learning for person re-identification: a survey and outlook[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 2872-2893.
[21] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[J].International Journal of Computer Vision, 2020, 128(2): 336-359.
[22] YE M, LAN X, WANG Z, et al. Bi-directional center-constrained top-ranking for visible thermal person re-identification[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 407-419.
[23] LI D, WEI X, HONG X, et al.Infrared-visible cross-modal person re-identification with an X modality[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020.
[24] CHOI S, LEE S, KIM Y, et al.Hi-CMD: hierarchical cross-modality disentanglement for visible-infrared person re-identification[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
[25] YE M, SHEN J B, CRANDALL D J, et al. Dynamic dual-attentive aggregation learning for visible-infrared person re-identification[C]//Proceedings of the European Conference on Computer Vision, 2020.
[26] CHEN Y, WAN L, LI Z, et al. Neural feature search for RGB-infrared person re-identification[C]//Proceedings of the 2021 IEEE/CVF Conference Vision and Pattern Recognition, 2021.
[27] LIU J L, SUN Y F, ZHU F, et al. Learning memory-augmented unidirectional metrics for cross-modality personre-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
[28] WANG P Y, ZHAO Z C, SU F, et al. Deep multi-patch matching network for visible thermal person re-identification[J].IEEE Transactions on Multimedia, 2020, 23: 1474-1488.
[29] DAI P Y, JI R R, WANG H B, et al. Cross-modality person re-identification with generative adversarial training[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, 2018.
[30] LU Y, WU Y, LIU B, et al. Cross-modality person re-identification with shared-specific feature transfer[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.