Unsupervised Person Re-Identification Method Based on Multi-Granularity Information Fusion

doi:10.3778/j.issn.1002-8331.2203-0498

Abstract

Abstract: Existing unsupervised person re-identification algorithms can only extract rough global features by residual network. However, as the dramatic increasing of persons, various poses and the complexity of the background in the database, these rough features are insufficient to represent different poses of persons, resulting in underfit of the model, which leads to the decline of recognition accuracy. Based on the analysis of the above problems, taking the information of both the spatial domain and the channel domain into account, an improved multi-granularity information fusion residual block（MgIFR block） is designed, in order to replace the conventional residual module in the ordinary residual network, thus a unsupervised person re-identification method based on MgIFR is proposed. In the spatial domain, the idea of self-attention mechanism is adopted in the construction of the residual block of MgIFR：firstly, the global feature of coarse-granularity is extracted by convolution; then, the fine-granularity global features, which are containing pixel-level context information, are obtained by combining the rough global features with the keys encoded at specific pixels in the image; thirdly, both of the coarse-grained and fine-grained global features are merged to generate the significant features of person postures. In the channel domain, the fusion of the significant pose features and the cross-layer features is weighted by channel attention mechanism, so as to build up the MgIFR block, which is able to improve the ability of the model to tackle different person postures. Experimental results illustrate that the performance of this algorithm on the accessible public datasets, especially those with more person gestures and more complicated background, is remarkable. Compared with the baseline model, the index values of rank-1 and mAP are improved by up to 9?percentage points and 10.7?percentage points, respectively. The proposed MgIFR block has superior ability in distinguishing person postures, effectively solving the problem of misjudgment caused by different person postures, and improving the accuracy of person re-identification recognition.

Key words: person re-identification, multi-granularity, residual block, self-attention mechanism, context information, feature fusion, unsupervised method

摘要： 现有的无监督行人重识别算法通过残差网络仅能提取粗略的全局特征，但是随着数据集中行人、姿态数目和背景复杂性的激增，这些特征表明行人不同姿态的能力不足，使得模型出现欠拟合，进而导致识别精度下降。基于对上述问题的分析，从空间域和通道域两方面考虑，设计了一种全新的多粒度信息融合的残差块（multi-granularity information fusion residual block，MgIFR block），替换残差网络中常规的残差模块，并以此提出了一种基于多粒度信息融合的无监督行人重识别方法。MgIFR模块在空间域上借鉴自注意力机制的思想，通过卷积提取粗粒度的全局特征；结合这些全局特征和图像中特定像素处编码的query，得到具有像素级上下文信息的细粒度全局特征，将具有粗粒度和细粒度的两种全局特征相结合，得到行人姿态的显著性特征；在通道域上，利用通道注意力机制，对输入的残差特征和跨层特征进行通道加权融合，最终得到具有多粒度信息融合的特征，以此来提高模型应对不同行人姿态的能力。实验结果表明，在现有公开数据集中，特别是行人数目姿态多和背景更加复杂的数据集上，相较于基线模型，Rank-1最高提升了9个百分点，mAP最高提升了10.7个百分点。提出的MgIFR模块具有更好的行人姿态的区分能力，有效解决了行人的不同姿态导致误判的问题，提高了行人重识别的准确率。

关键词: 行人重识别, 多粒度, 残差块, 自注意力机制, 上下文信息, 特征融合, 无监督方法

WEN Jing, ZHANG Fukang. Unsupervised Person Re-Identification Method Based on Multi-Granularity Information Fusion[J]. Computer Engineering and Applications, 2023, 59(13): 99-109.

温静, 张福康. 基于多粒度信息融合的无监督行人重识别方法[J]. 计算机工程与应用, 2023, 59(13): 99-109.

References

[1] 杨永胜，邓淼磊，李磊，等.基于深度学习的行人重识别综述[J].计算机工程与应用，2022，58（9）：51-66.
YANG Y S，DENG M L，LI L，et al.Overview of pedestrian re-identification based on deep learning[J].Computer Engineering and Applications，2022，58（9）：51-66.
[2] KOESTINGER M，HIRZER M，WOHLHART P，et al.Large scale metric learning from equivalence constraints[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition，2012：2288-2295.
[3] LI Z，CHANG S，LIANG F，et al.Learning locally-adaptive decision functions for verification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2013：3610-3617.
[4] MARTINEL N，MICHELONI C，FORESTI G L.Saliency weighted features for re-identification[C]//Proceedings of the European Conference on Computer Vision，2014：191-208.
[5] CUNNINGHAM P，CORD M，DELANY S J.Supervised learning[M]//Machine learning techniques for multimedia.Berlin，Heidelberg：Springer，2008：21-49.
[6] VARIOR R R，HALOI M，WANG G.Gated siamese convolutional neural network architecture for human re-identification[C]//Proceedings of the European Conference on Computer Vision，2016：791-808.
[7] VARIOR R R，SHUAI B，LU J，et al.A siamese long short-term memory architecture for human re-identification[C]//Proceedings of the European Conference on Computer Vision，2016：135-153.
[8] ZHANG X，LUO H，FAN X，et al.Alignedreid：surpassing human?level performance in re?identification[J].arXiv：1711.08184，2017.
[9] LYU J，CHEN W，LI Q，et al.Unsupervised cross-dataset re-identification by transfer learning of spatial-temporal patterns[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7948-7956.
[10] GE Y，ZHU F，CHEN D，et al.Self-paced contrastive learning with hybrid memory for domain adaptive object Re-ID[C]//Advances in Neural Information Processing Systems，2020：11309-11321.
[11] WANG D，ZHANG S.Unsupervised re-identification via multi-label classification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：10981-10990.
[12] WU J，YANG Y，LIU H，et al.Unsupervised graph association for re-identification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision，2019：8321-8330.
[13] LIN Y，DONG X，ZHENG L，et al.A bottom-up clustering approach to unsupervised re-identification[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：8738-8745.
[14] LIN Y，XIE L，WU Y，et al.Unsupervised re-identification via softened similarity learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：3390-3399.
[15] ZENG K，NING M，WANG Y，et al.Hierarchical clustering with hard-batch triplet loss for re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：13657-13665.
[16] DAI Z，WANG G，ZHU S，et al.Cluster contrast for unsupervised re-identification[J].arXiv：2103.11568，2021.
[17] LI Y，YAO T，PAN Y，et al.Contextual transformer networks for visual recognition[J].arXiv：2107.12292，2021.
[18] DAI Y，GIESEKE F，OEHMCKE S，et al.Attentional feature fusion[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision，2021：3560-3569.
[19] HE K，FAN H，WU Y，et al.Momentum contrast for unsupervised visual representation learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：9729-9738.
[20] ZHENG L，SHEN L，TIAN L，et al.Scalable re-identification：a benchmark[C]//Proceedings of the IEEE International Conference on Computer Vision，2015：1116-1124.
[21] RISTANI E，SOLERA F，ZOU R，et al.Performance measures and a data set for multi-target，multi-camera tracking[C]//Proceedings of the European Conference on Computer Vision，2016：17-35.
[22] WEI L，ZHANG S，GAO W，et al.Transfer GAN to bridge domain gap for re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：79-88.
[23] DENG J，DONG W，SOCHER R，et al.Imagenet：a large-scale hierarchical image database[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition，2009：248-255.
[24] ZHONG Z，ZHENG L，KANG G，et al.Random erasing data augmentation[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2020：13001-13008.
[25] ZHONG Z，ZHENG L，CAO D，et al.Re-ranking re-identification with k-reciprocal encoding[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：1318-1327.
[26] WANG Z，ZHANG J，ZHENG L，et al.Cycas：self-supervised cycle association for learning re-identifiable descriptions[C]//Proceedings of the European Conference on Computer Vision，2020：72-88.
[27] ZHAI Y，LU S，YE Q，et al.Ad-cluster：augmented discriminative clustering for domain adaptive re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：9021-9030.
[28] ZHONG Z，ZHENG L，LUO Z，et al.Invariance matters：exemplar memory for domain adaptive re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2019：598-607.
[29] LI M，ZHU X，GONG S.Unsupervised re-identification by deep learning tracklet association[C]//Proceedings of the European Conference on Computer Vision，2018：737-753.
[30] LI M，ZHU X，GONG S.Unsupervised tracklet re-identification[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，42（7）：1770-1782.
[31] GE Y，CHEN D，LI H.Mutual mean-teaching：pseudo label refinery for unsupervised domain adaptation on re-identification[J].arXiv：2001.01526，2020.