Anime Sketch Coloring with Enhanced Residual Swish Module

doi:10.3778/j.issn.1002-8331.2106-0415

Abstract

Abstract: Artificial anime sketch coloring costs time and vigor. Generative adversarial network（GAN） as a more popular network to generate color images in anime sketch coloring over the past few years has not achieved a certain effect on the direction actually, there are some problems such as color confusion, poor color gradient, unreasonable color sketch and so on. Thus, the proposed GAN network based on Swish function module can learn the details of the sketch better when the color fills, which avoids color confusion when the color exceeds the filled area. At the same time, it can directly carry out end-to-end training from anime sketch to color picture. The module can automatically color the sketch, generating color picture with rich color and clear texture. The experimental results show that this method has better coloring ability than the existing methods, and can obtain more realistic color pictures with better visual sense.

Key words: anime ketch coloring, Swish module, residual block, end-to-end

摘要： 人为的动漫线稿着色耗时费力，现有基于深度学习上色方法的着色效果并不是很好，比如颜色混淆不清晰、色彩梯度较差、颜色线条不合理等等。基于上述问题，提出了一种基于Swish激活函数残差增强模块的GAN网络，它可以更好地学习到颜色填充时线条的细节，避免颜色超过填充区域，造成颜色混淆。同时，该方法可以直接对线稿到彩色图片进行端到端的训练，仅仅输入线稿就可以自动地对线稿上色，生成颜色丰富纹理清晰的彩色图片。实验结果表明，该方法比现有方法具有更好的着色能力，可以获得更符合视觉感官、视觉效果更好的彩色图片。

关键词: 动漫素描着色, Swish连接模块, 残差模块, 端到端

FENG Yuting, LI Zhiwei. Anime Sketch Coloring with Enhanced Residual Swish Module[J]. Computer Engineering and Applications, 2023, 59(1): 221-227.

冯煜颋, 李志伟. Swish模块残差增强的动漫线稿上色方法[J]. 计算机工程与应用, 2023, 59(1): 221-227.

References

[1] SHELHAMER E，LONG J，DARRELL T.Fully convolutional networks for semantic segmentation[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39（4）：640-651.
[2] MIRZA M，OSINDERO S.Conditional generative adversarial nets[J].arXiv：1411.1784，2014.
[3] GOODFELLOW I J，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial networks[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge：MIT Press，2014：2672-2680.
[4] LIU Y F，QIN Z C，et al.Auto-painter：Cartoon image generation from sketch by using conditional wasserstein generative adversarial networks[J].Neurocomputing，2018，311：78-87.
[5] ZHANG L M，JI Y，LIN X.Style transfer for anime sketches with enhanced residual U-net and auxiliary classifier GAN[C]//Proceedings of Asian Conference on Pattern Recognition（ACPR 2017）.Los Alamitos，CA：IEEE Computer Society，2018：506-511.
[6] ISOLA P，ZHU J Y，ZHOU T H.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos：IEEE Computer Society，2017：5967-5976.
[7] WU J，ZHANG Y，WANG K.Skip connection U-net for white matter hyperintensities segmentation from MRI[J].IEEE Access，2019，7：155194-155202.
[8] LIU S，ZHANG X.Image decolorization combining local features and exposure features[J].IEEE Transactions on Multimedia，2019，21（10）：2461-2472.
[9] HE K，ZHANG X，REN S.Deep residual learning for image recognition[C]//Proceedings of 29th IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos，CA：IEEE Computer Society，2016：770-778.
[10] LIU S，ZHANG X.Automatic grayscale image colorization using histogram regression[J].Pattern Recognition Letters，2012，33：1673-1681.
[11] 邓森，徐进轩.自适应深度残差椒盐噪声滤除算法[J].计算机辅助设计与图形学学报，2020，32（8）：1248-1257.
DENG S，XU J X.Adaptive salt-and-pepper denoising based on deep residual network[J].Journal of Computer-Aided Design & Computer Graphics，2020，32（8）：1248-1257.
[12] LUCAS A，KATSAGGELOS A K，LOPEZ-TAPUIA S，et al.Generative adversarial networks and perceptual losses for video super-resolution[C]//Proceedings of 25th IEEE International Conference on Image Processing（ICIP）.Los Alamitos，CA：IEEE Computer Society，2018：51-55.
[13] DAI B，FIDLER S，URTASUN R，et al.Towards diverse and natural image descriptions via a conditional GAN[C]//Proceedings of the IEEE International Conference on Computer Vision.Los Alamitos：IEEE Computer Society，2017：2989-2998.
[14] GAN C，GAN Z，HE X D，et al.StyleNet：Generating attractive visual captions with styles[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Los Alamitos：IEEE Computer Society，2017：3137-3146.
[15] ZHANG R，ISOLA P，EFROS A A.Colorful image colorization[C]//Proceedings of the European Conference on Computer Vision.Heidelberg：Springer，2016：649-666.
[16] WANG Z，BOVIK A C，SHEIKH H R，et al.Image quality assessment：From error visibility to structural similarity[J].IEEE Transactions on Image Processing，2004，13（4）：600-612.
[17] ZHANG L，ZHANG L，MOU X，et al.FSIM：A feature similarity index for image quality assessment[J].IEEE Transactions on Image Processing，2011，20（8）：2378-2386.
[18] HEUSEL M，RAMSAUER H，UNTERTHINER T，et al.GANS trained by a two time-scale update rule converge to a nash equilibrium[C]//Advances in Neural Information Processing Systems，2017：6626-6637.