Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (21): 286-293.DOI: 10.3778/j.issn.1002-8331.2104-0130

• Engineering and Applications • Previous Articles     Next Articles

Virtual Try-On Method Introducing Differential Constraints and Adversarial Training Strategies

CHEN Yuefurong, LI Yi   

  1. College of Computer Science, Sichuan University, Chengdu 610065, China
  • Online:2022-11-01 Published:2022-11-01



  1. 四川大学 计算机学院,成都 610065

Abstract: In order to solve the problem of excessive clothing deformation and lack of texture in CP-VTON fittings, the skin color detection algorithm based on the ellipse model is used to correct the wrong division in the analysis area of the human body, and a differential constraint item defined according to the actual mesh is proposed. The thin plate spline transformation parameters learned and predicted by the constrained regression network to produce deformed clothing that meets the body shape of the target person. The U-Net-like network structure is used as the generator, the improved convolutional neural network is used as the discriminator. Generative confrontation training strategy is introduced to fuse the deformed clothing and the target person. Finally, the mask of the common area of the arm is obtained by re-recognition, and Poisson fusion is used to repair the hand feature information to improve the clarity of the hand. Experiments on VITON data set show that this method solves the original problems and achieves a better virtual fitting effect.

Key words: virtual try-on, generative adversarial networks, convolutional neural network, ellipse model

摘要: 为了解决CP-VTON在试衣中存在的服装形变过度和纹理缺失的问题,使用了基于椭圆模型的肤色检测算法修正人体解析区域中的错误划分,提出根据实际网格定义的差分约束项约束回归网络学习并预测的薄板样条变换参数,以产生符合目标人物身型的形变服装;使用类U-Net的网络结构作为生成器,改进的卷积神经网络作为判别器,引入生成对抗训练策略对形变服装和目标人物进行融合。最后,重识别得到手臂公共区域的蒙版,利用泊松融合修补手部特征信息,提高手部清晰度。在VITON的数据集上进行实验,结果表明该方法解决了原来存在的问题,取得了较好的虚拟试衣效果。

关键词: 虚拟试衣, 生成对抗网络, 卷积神经网络, 椭圆模型