Computer Engineering and Applications ›› 2024, Vol. 60 ›› Issue (15): 170-179.DOI: 10.3778/j.issn.1002-8331.2305-0042

• Graphics and Image Processing • Previous Articles     Next Articles

Image Arbitrary Style Transfer via Super-Resolution Reconstruction

TAN Run, TIAN Qichuan, LIAN Lu, ZHANG Xiaohang   

  1. College of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, Beijing 100044, China
  • Online:2024-08-01 Published:2024-07-30

融合超分辨率重构的图像任意风格迁移

谭润,田启川,廉露,张晓行   

  1. 北京建筑大学 电气与信息工程学院,北京 100044

Abstract: Image style transfer refers to the transformation of an ordinary photograph into an image with other artistic style effects. To address the problems of low definition and lack of texture details in the generated image due to the inability to reconstruct the resolution of the generated image in style transfer algorithms, an image arbitrary style transfer via super-resolution reconstruction is proposed. The multi-channel feature processing module added in the model enhances the feature expression by calculating the feature self-similarity, and a feature fusion module is proposed to improve the feature fusion effect. A feature decoder module is proposed to realize the image super-resolution reconstruction, in which features are fused several times to improve the stylized image quality. In the loss function, generative adversarial loss and whitening process are added to further improve the stylization effect. The experiment shows the model has great arbitrary style transfer effect, and the stylized image during resolution reconstruction has rich details and clear texture.

Key words: arbitrary style transfer, super-resolution reconstruction, generative adversarial network, self-attention mechanism

摘要: 图像风格迁移是指将一张普通照片转化为具有其他艺术风格效果的图像。针对风格迁移算法中无法重构生成图像的分辨率而造成生成图像清晰度低、纹理细节表现不丰富的问题,提出一种融合超分辨率重构的图像任意风格迁移模型。模型中加入的多支路特征处理模块通过计算特征的自相似性以增强特征的表达,提出新的特征融合模块以提升特征融合效果,提出特征解码模块来实现图像的超分辨率重构,并在其中多次进行特征融合以提升风格化图像的质量;在损失函数中加入生成对抗损失和白化处理来进一步提升风格化效果。实验表明,模型具有较好的任意风格迁移效果,分辨率重构后的风格化图像的细节丰富、纹理清晰。

关键词: 任意风格迁移, 超分辨率重构, 生成对抗网络, 自注意力机制