VTON-FG：通过图像边缘轮廓特征引导的虚拟试衣网络

doi:10.3778/j.issn.1002-8331.2312-0107

摘要/Abstract

摘要： 针对当前虚拟试衣方法在生成手臂和服装纹理细节方面普遍存在生成模糊和不完整的问题，基于CP-VTON+模型提出了一种名为VTON-FG的虚拟试衣网络。该网络引入了特征引导模块，在试穿模块中，通过提取服装边缘轮廓特征，并将不同尺度的特征信息融合到U-Net编码器的不同层次中，从而避免手臂信息和纹理信息丢失，引导图像生成更清晰的手臂图像和纹理特征。此外，在几何匹配模块中，为解决服装变形后可能导致的纹理畸变问题，引入了边缘轮廓图损失，从而提高了服装变形后的真实性。通过一系列消融实验，逐步验证了各模块在模型性能提升中的关键作用。实验结果表明，这些针对性的改进增强了模型的性能，从而证实了各模块在整体模型架构中的有效性和重要性。相对于CP-VTON+，该方法在结构相似度SSIM、感知相似度LPIPS和IS评价指标上分别提升了2.8%、19.4%和6.9%。

关键词: 虚拟试衣, 可控图像生成, 特征引导, 坐标注意力, 边缘检测

Abstract: In response to the prevalent issues of generating blurry and incomplete details in virtual garment try-on methods concerning the generation of arm and clothing texture details, a novel virtual try-on network named VTON-FG is proposed, based on the CP-VTON+ model. This network introduces a feature guided module. In the try-on module, it extracts clothing edge contour features and integrates features of different scales into various levels of the U-Net encoder. This approach aims to prevent the loss of arm and texture information, thereby guiding the generation of clearer arm images and texture features. Additionally, a geometric matching module is incorporated to address potential texture distortions arising from clothing deformation, introducing an edge contour loss to enhance the authenticity of clothing deformation. A series of ablation experiments are conducted to gradually verify the crucial role of each module in improving model performance. The experimental results indicate that these targeted improvements have enhanced the performance of model, thus confirming the effectiveness and importance of each module within the overall model architecture. Relative to CP-VTON+, this method exhibits enhancements of 2.8%, 19.4%, and 6.9% in structural similarity (SSIM), perceptual similarity (LPIPS), and IS evaluation metrics, respectively.

Key words: virtual try-on, controllable image generation, feature guidance, coordinate attention, edge detection

谭台哲, 陈宏才, 杨卓. VTON-FG：通过图像边缘轮廓特征引导的虚拟试衣网络[J]. 计算机工程与应用, 2025, 61(9): 255-262.

TAN Taizhe, CHEN Hongcai, YANG Zhuo. VTON-FG： Virtual Try-on Network Guided by Image Edge Contour Features[J]. Computer Engineering and Applications, 2025, 61(9): 255-262.

参考文献

[1] HAN X T, WU Z X, WU Z, et al. VITON: an image-based virtual try-on network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7543-7552.
[2] BELONGIE S, MALIK J, PUZICHA J. Shape matching and object recognition using shape contexts[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(4): 509-522.
[3] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014.
[4] WANG B, ZHENG H, LIANG X, et al. Toward characteristic-preserving image-based virtual try-on network[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 589-604.
[5] YANG H, ZHANG R, GUO X, et al. Towards photo-realistic virtual try-on by adaptively generating-preserving image content[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 7850-7859.
[6] MINAR M R, TUAN T T, AHN H, et al. CP-VTON+: clothing shape and texture preserving image-based virtual try-on[C]//Proceedings of the IEEE 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020: 10-14.
[7] DHARIWAL P, NICHOL A. Diffusion models beat GANs on image synthesis[C]//Advances in Neural Information Processing Systems, 2021: 8780-8794.
[8] SHA T, ZHANG W, SHEN T, et al. Deep person generation: a survey from the perspective of face, pose, and cloth synthesis[J]. ACM Computing Surveys, 2023, 55(12): 1-37.
[9] YANG H, ZHANG R, GUO X, et al. Towards photo-realistic virtual try-on by adaptively generating-preserving image content[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 7850-7859.
[10] GE C, SONG Y, GE Y, et al. Disentangled cycle consistency for highly-realistic virtual try-on[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 16928-16937.
[11] HE S, SONG Y Z, XIANG T. Style-based global appearance flow for virtual try-on[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022: 3470-3479.
[12] KARRAS T, LAINE S, AILA T. A style-based generator architecture for generative adversarial networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 4401-4410.
[13] GAO Y, WEI F, BAO J, et al. High-fidelity and arbitrary face editing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 16115-16124.
[14] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1125-1134.
[15] ZHANG L, RAO A, AGRAWALA M. Adding conditional control to text-to-image diffusion models[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision,2023: 3836-3847.
[16] ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022: 10684-10695.
[17] MOU C, WANG X, XIE L, et al. T2I-adapter: learning adapters to dig out more controllable ability for text-to-image diffusion models[J]. arXiv:2302.08453, 2023.
[18] CANNY J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986 (6): 679-698.
[19] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1125-1134.
[20] LIN M, CHEN Q, YAN S. Network in network[J]. arXiv:1312.4400, 2013.
[21] RUMELHART D E, HINTON G E, WILLIAMS R J. Learning representations by back-propagating errors[J]. Nature, 1986, 323(6088): 533-536.
[22] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention(MICCAI 2015), Munich, Germany, October 5-9, 2015: 234-241.
[23] GATYS L A, ECKER A S, BETHGE M. A neural algorithm of artistic style[J]. arXiv:1508.06576, 2015.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409.
1556, 2014.
[25] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[26] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 586-595.
[27] SALIMANS T, GOODFELLOW I, ZAREMBA W, et al. Improved techniques for training gans[C]//Advances in Neural Information Processing Systems, 2016.