Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (14): 167-176.DOI: 10.3778/j.issn.1002-8331.2104-0083

• Graphics and Image Processing • Previous Articles     Next Articles

New GAN-Based Partial Realistic Anime Image Style Transfer

SUN Tianpeng, ZHOU Ningning, HUANG Guofang   

  1. 1.School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
    2.NARI Technology Co., Ltd., Nanjing 211106, China
  • Online:2022-07-15 Published:2022-07-15

新的基于GAN的局部写实感漫画图像风格迁移

孙天鹏,周宁宁,黄国方   

  1. 1.南京邮电大学 计算机学院,南京 210023
    2.国电南瑞科技股份有限公司,南京 211106

Abstract: Generative adversarial network(GAN) is one of the research hotspots in computer vision to transfer the style of images from real world to high quality animation style. The popular AnimeGAN and CartoonGAN anime generating networks have suffered from serious detail loss and color distortion during image migration. In this paper, a new ExpressionGAN is proposed to solve the problem of serious detail loss in AnimeGAN transfer images by introducing the SE-Residual Block, the face detection mechanism and optimizing the loss function. By adding DSCONV(distributed shift convolution), it is proposed that SceneryGAN speeds up the training and eliminates the ambiguous pixel blocks in the CartoonGAN migration images. The image fusion boundary is optimized by convolution. At the same time, a new partial realistic cartoon model is proposed, which deals with and integrates the original image characters and environment respectively. The experimental results show that, compared with AnimeGAN and CartoonGAN, the proposed method has significantly improved the training speed, anime image generation quality and local realism of anime images.

Key words: image style transfer, generative adversarial network, anime style, partial sense of reality, AnimeGAN, CartoonGAN

摘要: 利用生成对抗网络对图像进行风格迁移,将真实世界的图像直接转换为高品质动漫风格,是当今计算机视觉的研究热点之一。针对目前流行的AnimeGAN和CartoonGAN漫画生成对抗网络在图像迁移中存在细节丢失严重、色彩失真等问题。通过引入SE-Residual Block(挤压激励残差块)、漫画脸部检测机制并优化损失函数提出全新的ExpressionGAN解决了AnimeGAN迁移图像细节丢失严重的问题。通过加入DSConv(分布偏移卷积)提出SceneryGAN 加快了训练速度并消除了CartoonGAN迁移图像中的歧义像素块。通过卷积优化了图像的融合边界。同时,提出了一种新的对原始图像人物和环境分别处理并融合的局部写实主义漫画模型。实验结果表明,与AnimeGAN和CartoonGAN相比,该方法在训练速度、漫画图像生成质量和图像局部写实感方面都有了明显的提升。

关键词: 图像风格迁移, 生成对抗网络, 动漫风格, 局部真实感, AnimeGAN, CartoonGAN