Research on Person Re-Identification Algorithm Based on Multi-Pose Image Generation

doi:10.3778/j.issn.1002-8331.2107-0156

Abstract

Abstract: The high similarities of different person and great diversities in appearance of the same person pose grand challenges to person re-identification（ReID）. Generative adversarial networks（GAN） can synthesize new images and are considered to be the main technical means to solve the problem of pose variations. This paper proposes a multi-pose image generation algorithm for person ReID, which adopts GAN to generate person images with different poses. By eliminating the influence of pose through normalization, the overall performance of person ReID can be improved greatly. The proposed person ReID algorithm includes three parts：multi-pose person image generation, feature extraction and fusion for person in different pose, distance measurement and reranking. Experiments on the Market-1501 dataset and DukeMTMC-ReID dataset demonstrate the validity of the proposed algorithm and show the superiority of multi-pose image generation for improving person ReID through comparisons with the state-of-the-art methods. At last, it shows that the features of the generated images and the original images are complementary.

Key words: person re-identification, pose estimation, generative adversarial networks, pose normalization

摘要： 不同行人的高度相似性以及相同行人外观姿态的差异性，使得不同摄像头下的行人重识别面临严峻的挑战。生成对抗网络可以合成新的图像，被认为是解决行人姿态变化的主要技术手段。提出一种基于多姿态图像生成的行人重识别算法，利用生成对抗网络生成不同姿态的行人图像，通过归一化消除姿态的影响，从而大幅度提升行人重识别的整体性能。该行人重识别算法包括多姿态行人图像生成、不同姿态的行人特征提取与融合、距离度量和重排序三部分内容。在Market-1501数据集和DukeMTMC-ReID数据集上的实验证实了所提出算法的有效性，通过与state-of-the-art行人重识别方法比较，展示了多姿态图像生成方法在行人重识别任务中的优越性，同时表明生成行人图像的特征与原始图像的特征是相互补充的。

关键词: 行人重识别, 姿态估计, 生成对抗网络, 姿态归一化

ZHANG Haiyan, ZHANG Fukai, YUAN Guan, LI Yingying. Research on Person Re-Identification Algorithm Based on Multi-Pose Image Generation[J]. Computer Engineering and Applications, 2023, 59(2): 143-152.

张海燕, 张富凯, 袁冠, 李莹莹. 多姿态图像生成的行人重识别算法研究[J]. 计算机工程与应用, 2023, 59(2): 143-152.

References

[1] ZHENG Z，ZHENG L，YANG Y.A discriminatively learned CNN embedding for person reidentification[J].ACM Transactions on Multimedia Computing Communications and Applications，2018，14（1）：1-20.
[2] GOODFELLOW I J，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial networks[J].arXiv：1406.2661，2014.
[3] MA L，JIA X，SUN Q，et al.Pose guided person image generation[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems，2017：405-415.
[4] ZHAO B，WU X，CHENG Z Q，et al.Multi-view image generation from a single-view[J].arXiv：1704.04886v4，2017.
[5] ZHENG L，SHEN L，TIAN L，et al.Scalable person re-identification：a benchmark[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），Santiago，2015：1116-1124.
[6] MA L，LIU H，HU L，et al.Orientation driven bag of appearances for person re-identification[J].arXiv：1605.02464，2016.
[7] LI W，ZHAO R，XIAO T，et al.DeepReID：deep filter pairing neural network for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Columbus，OH，USA，2014：152-159.
[8] RISTANI E，SOLERA F，ZOU R，et al.Performance measures and a data set for multi-target，multi-camera tracking[C]//Proceedings of the European Conference on Computer Vision（ECCV），Amsterdam，The Netherlands，2016：17-35.
[9] ZHENG Z，ZHENG L，YANG Y.Unlabeled samples generated by GAN improve the person re-identification baseline in vitro[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），Venice，Italy，2017：3774-3782.
[10] WEI L，ZHANG S，GAO W，et al.Person transfer GAN to bridge domain gap for person re-identification[J].arXiv：1711.08565v2，2017.
[11] ZHANG J，NIU L，ZHANG L.Person re-identification with reinforced attribute attention selection[J].IEEE Transactions on Image Processing，2020，30：603-616.
[12] FAYYAZ M，YASMIN M，SHARIF M，et al.Person re-identification with features-based clustering and deep features[J].Neural Computing and Applications，2020，32（14）：10519-10540.
[13] WU W，TAO D，LI H，et al.Deep features for person re-identification on metric learning[J].Pattern Recognition，2021，110：107424.
[14] ZHONG W，JIANG L，ZHANG T，et al.A part-based attention network for person re-identification[J].Multimedia Tools and Applications，2020，79（31/32）：22525-22549.
[15] CHONG Y，PENG C，ZHANG J，et al.Style transfer for unsupervised domain-adaptive person re-identification[J].Neurocomputing，2021，422：314-321.
[16] WANG C，HAN H，SHANG X，et al.A new deep learning method based on unsupervised domain adaptation and re-ranking in person re-identification[J].International Journal of Pattern Recognition and Artificial Intelligence，2020，34（12）：2052011.
[17] YAO R，GAO C，XAI S，et al.GAN-based person search via deep complementary classifier with center-constrained Triplet loss[J].Pattern Recognition，2020，104：107350.
[18] ZHANG C，ZHU L，ZHANG S，et al.PAC-GAN：an effective pose augmentation scheme for unsupervised cross-view person re-identification[J].Neurocomputing，2020，387：22-39.
[19] REN C X，LIANG B，GE P，et al.Domain adaptive person re-identification via camera style generation and label propagation[J].IEEE Transactions on Information Forensics and Security，2020，15：1290-1302.
[20] ZHANG C，WU L，WANG Y.Crossing generative adversarial networks for cross-view person re-identification[J].Neurocomputing，2019，340：259-269.
[21] CHEN F，WANG N，TANG J，et al.Self-supervised data augmentation for person re-identification[J].Neurocomputing，2020，415：48-59.
[22] ZHE C，GINES H，TOMAS S，et al.OpenPose：realtime multi-person 2D pose estimation using part affinity fields[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43（1）：172-186.
[23] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Las Vegas，NV，USA，2016：770-778.
[24] HUANG G，LIU Z，MAATEN L V D，et al.Densely connected convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Honolulu，HI，USA，2017：2261-2269.
[25] KINGMA D，BA J.Adam：a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations（ICLR），San Diego，CA，USA，2015：1-15.
[26] MANDT S，HOFFMAN M D，BLEI D M.Stochastic gradient descent as approximate Bayesian inference[J].Machine Learning Research，2017，18：1-35.
[27] DENG W，ZHENG L，YE Q，et al.Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），Salt Lake City，UT，USA，2018：994-1003.
[28] LI D，CHEN X，ZHANG Z，et al.Learning deep context-aware features over body and latent parts for person re-identification[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Honolulu，HI，USA，2017：7398-7407.
[29] SUN Y，ZHENG L，DENG W，et al.SVDNet for pedestrian retrieval[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），Venice，Italy，Oct 22-29，2017：3820-3828.
[30] ZHENG Z，ZHENG L，YANG Y.Pedestrian alignment network for large-scale person re-identification[J].IEEE Transactions on Circuits and Systems for Video Technology，2019，29（10）：3037-3045.
[31] LIN Y，ZHENG L，ZHENG Z，et al.Improving person re-identification by attribute and identity learning[J].Pattern Recognition，2019，9：151-161.
[32] WEI L，ZHANG S，YAO H，et al.GLAD：global-local-alignment descriptor for pedestrian retrieval[C]//Proceedings of the 25th ACM International Conference on Multimedia（MM），2017：420-428.
[33] LIU J，NI B，YAN Y，et al.Pose transferrable person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），Salt Lake City，UT，USA，2018：4099-4108.
[34] HUANG Y，XU J，WU Q，et al.Multi-pseudo regularized label for generated data in person re-identification[J].IEEE Transactions on Image Processing，2018，28（3）：1391-1403.

[35] QIAN X，FU Y，WANG W，et al.Pose-normalized image generation for person re-identification[J].arXiv：1712.02225，2017.

[36] GE Y，LI Z，ZHAO H，et al.FD-GAN：pose-guided feature distilling GAN for robust person re-identification[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems（NIPS），2018：1230-1241.