基于生成对抗网络的图像动漫风格化

doi:10.3778/j.issn.1002-8331.2109-0484

摘要/Abstract

摘要： 目前的卡通风格图片生成方法仍然存在局限，如色彩不真实、图片局部细节处理不到位等，要想快速将输入图片转换为动漫的风格输出还需要结合深度学习进行研究。基于生成对抗网络的思想，提出了一种动漫风格化编码的生成对抗网络，将输入的图像风格转变为宫崎骏动画电影的风格。网络结构加入自适应实例归一化层（AdaIN）模块和多层感知机（MLP）模块，得到很大优化，同时提高实验效果。在损失函数部分，引入图像感知相似性（lpips）作为内容损失函数，二分类交叉熵（binary cross entropy）损失函数（BCELoss）作为对抗损失函数。实验结果表明，该网络对于动漫化图片起到了很好的效果，FID分数72，能够灵活适用于各种类型的图片动漫化。

关键词: 风格迁移, 图像到图像转换, 生成对抗网络, 动漫化图片, 感知损失

Abstract: The current cartoon style image generation methods still have limitations, such as the unrealistic color, inadequate processing of local details of the picture, and so on. In order to quickly convert the input image into the style of animation, you need to combine deep learning. Based on the idea of generative adversarial network, the proposed algorithm is a generative adversarial network of animation stylized coding, which transforms the input image style into Hayao Miyazaki’s animated film style. The network structure has been greatly optimized by adding the adaptive instance normalization layer（AdaIN） module and the multi-layer perceptron（MLP） module, while improving the experimental effect. In the loss function part, learned perceptual image patch similarity（LPIPS） is introduced as the content loss function, and the binary cross entropy loss function（BCELoss） is used as the adversarial loss function. Experimental results show that the network has a very good effect on animated pictures, with an FID score of 72, which can be flexibly applied to various types of pictures animating.

Key words: style transfer, image to image translation, generative adversarial network（GAN）, animation picture, perceptual loss

王一凡, 赵乐义, 李毅. 基于生成对抗网络的图像动漫风格化[J]. 计算机工程与应用, 2022, 58(18): 104-110.

WANG Yifan, ZHAO Leyi, LI Yi. Image Animation Stylization Based on Generative Adversarial Network[J]. Computer Engineering and Applications, 2022, 58(18): 104-110.

参考文献

[1] ZHU J Y，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//International Conference on Computer Vision，2017.
[2] HUANG X，LIU M Y，BELONGIE S，et al.Multimodal unsupervised image to image translation[C]//Proceedings of the European Conference on Computer Vision，2018：172-189.
[3] YI Z，ZHANG H，TAN P，et al.DualGAN：unsupervised dual learning for image-to-image translation[J].arXiv：1704.02510，
2017.
[4] HUANG X，BELONGIE S.Arbitrary style transfer in real-time with adaptive instance normalization[C]//Proceedings of the IEEE International Conference on Computer Vision，2017：1501-1510.
[5] GOODFELLOW I J，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Conference on Neural Information Processing Systems.[S.l.]：MIT Press，2014：2672-2680.
[6] ISOLA P，ZHU J Y，ZHOU T，et al.Image-to-image translation with conditional adversarial networks[C]//IEEE Conference on Computer Vision and Pattern Recognition，2017.
[7] WANG T C，LIU M Y，ZHU J Y，et al.High resolution image synthesis and semantic manipulation with conditional gans[C]//IEEE Conference on Computer Vision and Pattern Recognition，2018.
[8] WANG X R，YU J Z.Learning to cartoonize using white-box cartoon representations[C]//IEEE Conference on Computer Vision and Pattern Recognition，2020.
[9] CHEN Y，LAI Y K，LIU Y J.CartoonGAN：generative adversarial networks for photo cartoonization[C]//IEEE Conference on Computer Vision and Pattern Recognition，2018.
[10] GATYS L A，ECKER A S，BETHGE M.Image style transfer using convolutional neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：2414-2423.
[11] IOFFE S，SZEGEDY C.Batch normalization：accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning，2015.
[12] ULYANOV D，VEDALDI A，LEMPITSKY V.Improved texture networks：maximizing quality and diversity in feed-forward stylization and texture synthesis[C]//IEEE Conference on Computer Vision and Pattern Recognition，2017.
[13] LI C，WAND M.Combining Markov random fields and convolutional neural networks for image synthesis[C]//IEEE Conference on Computer Vision and Pattern Recognition，2016.
[14] ZHANG R，ISOLA P，EFROS A A，et al.The unreasonable effectiveness of deep features as a perceptual metric[C]//IEEE Conference on Computer Vision and Pattern Recognition，2018.
[15] DUMOULIN V，SHLENS J，KUDLUR M.A learned representation for artistic style[J].arXiv：1610.07629，2016.
[16] JOHNSON J，ALAHI A，LI F F.Perceptual losses for real-time style transfer and super-resolution[C]//International Conference on Computer Vision，2016.
[17] HEUSEL M，RAMSAUER H，UNTERTHINER T，et al.GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]//Conference on Neural Information Processing Systems.[S.l.]：MIT Press，2017：6627-6638.