Virtual Try-On Method Introducing Differential Constraints and Adversarial Training Strategies

doi:10.3778/j.issn.1002-8331.2104-0130

Abstract

Abstract: In order to solve the problem of excessive clothing deformation and lack of texture in CP-VTON fittings, the skin color detection algorithm based on the ellipse model is used to correct the wrong division in the analysis area of the human body, and a differential constraint item defined according to the actual mesh is proposed. The thin plate spline transformation parameters learned and predicted by the constrained regression network to produce deformed clothing that meets the body shape of the target person. The U-Net-like network structure is used as the generator, the improved convolutional neural network is used as the discriminator. Generative confrontation training strategy is introduced to fuse the deformed clothing and the target person. Finally, the mask of the common area of the arm is obtained by re-recognition, and Poisson fusion is used to repair the hand feature information to improve the clarity of the hand. Experiments on VITON data set show that this method solves the original problems and achieves a better virtual fitting effect.

Key words: virtual try-on, generative adversarial networks, convolutional neural network, ellipse model

摘要： 为了解决CP-VTON在试衣中存在的服装形变过度和纹理缺失的问题，使用了基于椭圆模型的肤色检测算法修正人体解析区域中的错误划分，提出根据实际网格定义的差分约束项约束回归网络学习并预测的薄板样条变换参数，以产生符合目标人物身型的形变服装；使用类U-Net的网络结构作为生成器，改进的卷积神经网络作为判别器，引入生成对抗训练策略对形变服装和目标人物进行融合。最后，重识别得到手臂公共区域的蒙版，利用泊松融合修补手部特征信息，提高手部清晰度。在VITON的数据集上进行实验，结果表明该方法解决了原来存在的问题，取得了较好的虚拟试衣效果。

关键词: 虚拟试衣, 生成对抗网络, 卷积神经网络, 椭圆模型

CHEN Yuefurong, LI Yi. Virtual Try-On Method Introducing Differential Constraints and Adversarial Training Strategies[J]. Computer Engineering and Applications, 2022, 58(21): 286-293.

陈玥芙蓉, 李毅. 引入差分约束和对抗训练策略的虚拟试衣方法[J]. 计算机工程与应用, 2022, 58(21): 286-293.

References

[1] GOODFELLOW I，POUGET-A BADIE J，MIRZA M，et al.Generative adversarial nets[C]//Advances in Neural Information Processing Systems，2014：2672-2680.
[2] JETCHEV N，BERGMANN U.The conditional analogy GAN：swapping fashion articles on people images[C]//2017 IEEE International Conference on Computer Vision Workshop（ICCVW），2017.
[3] PANDEY N，SAVAKIS A.Poly-GAN：multi-conditioned GAN for fashion synthesis[J].Neurocomputing，2020，414：356-364.
[4] HAN X T，WU Z X，WU Z，et al.VITON：an image-based virtual try-on network[C]//IEEE International Conference on Computer Vision，2018：7543-7552.
[5] WANG B，ZHENG H，LIANG X，et al.Toward characteristic-preserving image-based virtual try-on network[C]//15th European Conference，Munich，Germany，September 8-14，2018.
[6] BOOKSTEIN F L.Principal warps：thin-plate splines and the decompositions of deformations[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，1989，11（6）：567-585.
[7] LEE H J，LEE R，KANG M，et al.LA-VITON：a network for looking-attractive virtual try-on[C]//International Conference on Computer Vision（ICCV），2019.
[8] DONG H，LIANG X，WANG B，et al.Towards multi-pose guided virtual try-on network[J].2019 IEEE/CVF International Conference on Computer Vision（ICCV），2019.
[9] RONNEBERGER O，FISCHER P，BROX T.U-net：convolutional networks for biomedical image segmentation[C]//International Conference on Medical-Image Computing and Computer Assisted Intervention.Cham：Springer，2015：234-241.
[10] RADFORD A，METZ L，CHINTALA S.Unsupervised representation learning with deep convolutional generative adversarial networks[J].arXiv：1511.06434，2015.
[11] CAO Z，HIDALGO G，SIMON T，et al.Open-pose：realtime multi-person 2D pose estimation using part affinity fields[EB/OL].[2020-05-10].https：//openaccess.thecvf.com/content_cvpr_2017/papers/Cao_Realtime_Multi-Person_2D_
CVPR_2017_paper.pdf.
[12] KE G，LIANG X，ZHANG D，et al.Look into person：self-supervised structure-sensitive learning and a new benchmark for human parsing[J].arXiv：1703.05446，2017.
[13] 高建坡，王煜坚，杨浩，等.一种基于KL变换的椭圆模型肤色检测方法[J].电子与信息学报，2007，29（7）：1739-1743.
GAO J P，WANG Y J，YANG H，et al.An ellipse model based on KL transformation for skin color detection[J].Journal of Electronics and Information Technology，2007，29（7）：1739-1743.
[14] ROCCO I， SIVIC J.Convolutional neural network architecture for geometric matching[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，41（11）：2553-2567.
[15] ULYANOV D，VEDALDI A，LEMPITSKY V.Instance normalization：the missing ingredient for fast stylization[J].arXiv：1607.08022，2016.
[16] JOHNOSON J，ALAHI A，LI F F.Perceptual losses for real-time style transfer and super-resolution[C]//European Conference on Computer Vision.Cham：Springer，2016：694-711.
[17] 金耀，熊宇龙，周泳全，等.复用拉普拉斯算子的高效网格融合方法[J].软件学报，2019，30（12）：3862-3875.
JIN Y，XIONG Y L，ZHOU Y Q，et al.Efficient mesh merging method with reusable Laplace operator[J].Journal of Software，2019，30（12）：3862-3875.