Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (6): 146-154.DOI: 10.3778/j.issn.1002-8331.2110-0107

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Content Structure Preserved Image Style Transfer Method

WANG Xiaoming, MAO Yushi, XU Bin, WANG Zilei   

  1. 1.Electric Power Research Institute, State Grid Anhui Electric Power Co., Ltd., Hefei 230601, China
    2.Institute of Advanced Technology, University of Science and Technology of China, Hefei 230000, China
  • Online:2023-03-15 Published:2023-03-15

内容结构保持的图像风格迁移方法

王小明,毛语实,徐斌,王子磊   

  1. 1.国网安徽省电力有限公司 电力科学研究院,合肥 230601
    2.中国科学技术大学 先进技术研究院,合肥 230000

Abstract: Most existing style transfer algorithms realize the conversion task from art works to real scenes, such as photos. Under this experiment setting, the content structure of the image may change to a certain extent. However, this change is not conducive to the completion of downstream tasks, so it is not available for most industrial scenes. Therefore, a new structure preserving style transfer method is proposed, which contains a frequency domain constrained image generation module and a semantic matching module based on memory bank. The former is used to ensure the consistency of the overall semantic structure of the image before and after transfer, while the latter ensures the semantic and style matching, so that the internal structure of the object is consistent. By comparing the frequency domain similarity between the original image and the migrated image and the similarity between the generated image and the corresponding category in memory bank, the purpose of keeping the image structure unchanged is achieved. In order to verify the effect this method produced in the industrial scene, the photovoltaic panel dataset is used. The proposed style transfer algorithm can not only complete the style conversion, but also better resolve the problem of image structure deformation, so as to meet the needs of subsequent tasks.

Key words: image style transfer, generative adversarial network, computer vision

摘要: 现有的风格迁移算法大多是实现艺术作品到真实场景,如照片等的转换任务。在这种任务设定下,图像内容的边界等结构信息可能会出现一定程度的变化,然而这种变化不利于下游任务的完成,因而不适用于大部分工业场景。为此提出一种新的结构保持的风格迁移方法,该方法分为频域约束的图像生成模块和基于memory bank机制的语义匹配模块。前者用以保证转换前后图像的整体语义结构一致性,而后者保证了图像的语义与风格匹配,从而物体内部结构一致。为了验证该方法在工业场景中的迁移效果,采用光伏板数据集,在虚拟到真实的迁移方向上,所提出的风格迁移算法能在完成风格转换的同时较好地解决图像结构形变问题,进而满足后续任务的需求。

关键词: 图像风格迁移, 生成对抗网络, 计算机视觉