Image Style Transfer Method for Industrial Inspection

doi:10.3778/j.issn.1002-8331.2205-0583

Abstract

Abstract: For the inspection tasks in industrial scenarios such as electric power, it is necessary to convert the virtual images generated by virtual simulation into real style to assist the next photovoltaic panel positioning and defect detection. However, most of the traditional image style transfer methods focus on the translation between artistic image and real scene. Due to the characteristics of artistic image itself, to a certain extent, the content structure of image may change before and after translation. This change will have a negative impact on the completion of downstream tasks, so it can hardly be applied in industrial scenarios. A contrastive learning-based visual style transfer method is proposed in this research. The created images can be closer to the target domain in the deep features by comparing the features of the same and different image blocks on many layers of features, while restricting the image before and after the transfer. The image content is uniform. The experimental results suggest that the strategy proposed in this study can reduce model training time and computational overhead while assuring that the key content of image is not warped or absent.

Key words: image style transfer, generative adversarial network, virtual reality, contrastive learning

摘要： 对于电力等工业场景中的巡检任务，需要将虚拟仿真生成的虚拟图像转换为真实的风格，辅助接下来的设备定位和设备缺陷检测工作。然而传统的图像风格迁移方法大多着眼于艺术图像与现实场景的转换，由于艺术图像本身的特性，在迁移前后图像的内容结构可能会出现一定程度的变化，这种变化会对下游任务的性能造成负面影响，因而无法直接应用。因此提出一种基于对比学习的图像风格迁移方法，通过在多个层次的特征上分别对比相同与不同位置图像块的特征，能够使生成图像在深层特征上向目标域靠近，同时约束迁移前后的图像内容上保持一致。实验表明，所提出的方法能够在保证图像主体内容不发生变形和缺失的同时，节省模型训练过程中的计算开销。

关键词: 图像风格迁移, 生成对抗网络, 虚拟仿真, 对比学习

ZHU Zhongxian, MAO Yushi, CAI Kewei, LIU Wentao, PU Daojie, DU Yao, WANG Zilei. Image Style Transfer Method for Industrial Inspection[J]. Computer Engineering and Applications, 2023, 59(18): 234-241.

朱仲贤, 毛语实, 蔡科伟, 刘文涛, 蒲道杰, 杜瑶, 王子磊. 面向工业巡检的图像风格迁移方法[J]. 计算机工程与应用, 2023, 59(18): 234-241.

References

[1] 高文超，任圣博，田驰，等.多层次生成对抗网络的动画头像生成方法研究[J].计算机工程与应用，2022，58（9）：230-237.
GAO W C，REN S B，TIAN C，et al.Research on method of animated avatar generation based on multi-level generative adversarial networks[J].Computer Engineering and Applications，2022，58（9）：230-237.
[2] 王杨，郁振鑫，卢嘉，等.多区域差异化的图像风格迁移算法[J].计算机工程与应用，2021，57（12）：211-215.
WANG Y，YU Z X，LU J，et al.Multi-region differential image style transfer algorithm[J].Computer Engineering and Applications，2021，57（12）：211-215.
[3] 李慧，万晓霞.深度卷积神经网络下的图像风格迁移算法[J].计算机工程与应用，2020，56（2）：176-183.
LI H，WAN X X.Image style transfer algorithm under deep convolutional neural network[J].Computer Engineering and Applications，2020，56（2）：176-183.
[4] CHEN X，DUAN Y，HOUTHOOFT R，et al.InfoGAN：interpretable representation learning by information maximizing generative adversarial nets[C]//Proceedings of the 29th International Conference on Neural Information Processing Systems，2016.
[5] LIU M Y，BREUEL T，KAUTZ J.Unsupervised image-to-image translation networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems，2017.
[6] LI M，HUANG H，MA L，et al.Unsupervised image-to-image translation with stacked cycle-consistent adversarial networks[C]//Proceedings of the European Conference on Computer Vision，2018：184-199.
[7] ROY S，SIAROHIN A，SANGINETO E，et al.TriGAN：image-to-image translation for multi-source domain adaptation[J].Machine Vision and Applications，2021，32（1）：1-12.
[8] LI S，GUNEL S，OSTREK M，et al.Deformation aware unpaired image translation for pose estimation on laboratory animals[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2020：13158-13168.
[9] 王大方，杜京东，曹江，等.面向自动驾驶数据生成的风格迁移网络研究[J].汽车工程，2022，44（5）：684-690.
WANG D F，DU J D，CAO J，et al.Research on style transfer network for autonomous driving data generation[J].Automotive Engineering，2022，44（5）：684-690.
[10] GOODFELLOW I，POUGETABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems，2014.
[11] LEE H Y，TSENG H Y，HUANG J B，et al.Diverse image-to-image translation via disentangled representations[C]//Proceedings of the European Conference on Computer Vision，2018：35-51.
[12] CHANG H Y，WANG Z，CHUANG Y Y.Domain-specific mappings for generative adversarial style transfer[C]//European Conference on Computer Vision，2020：573-589.
[13] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2017：2223-2232.
[14] HAN J，SHOEIBY M，PETERSSON L，et al.Dual contrastive learning for unsupervised image-to-image translation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：746-755.
[15] PARK T，EFROS A A，ZHANG R，et al.Contrastive learning for unpaired image-to-image translation[C]//European Conference on Computer Vision，2020：319-345.
[16] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2016：770-778.
[17] ISOLA P，ZHU J Y，ZHOU T，et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2017：1125-1134.
[18] SZEGEDY C，VANHOUCKE V，IOFFE S，et al.Rethinking the inception architecture for computer vision[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2016：2818-2826.
[19] ZHANG R，ISOLA P，EFROS A A，et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：586-595.