Personalized Handwritten Chinese Character Generation Method for Unsupervised Image Translation

doi:10.3778/j.issn.1002-8331.2010-0033

Abstract

Abstract: Chinese characters have a large number of characters, most of the researches focus on its recognition and classification. There are few researches on automatic generation of Chinese characters, especially in the absence of a large number of paired characters datasets. This model uses the unpaired Chinese characters datasets to learn the characteristics of personalized handwritten Chinese characters and transform the formula of generating Chinese handwritten characters into a problem of mapping from the existing standard printing font to the personalized handwritten Chinese characters style. Based on the unsupervised image translation model, attention mechanism and adaptive normalization layer are added to enhance the content and the style of personalized Chinese characters image generation and the ability of the discriminator network is improved by modifying the loss function. Experiments are conducted on the CASIA-HWDB handwritten Chinese character dataset and the Lantingxu calligraphy dataset, and the effectiveness of the method is verified by comparing the evaluation indexes of the content accuracy and the style differences.

Key words: generative adversarial network, image-to-image translation, unsupervised learning, generation of handwritten Chinese characters

摘要： 由于汉字拥有大量的字符，大多数对汉字的研究主要集中在汉字的识别和分类问题上，对于生成汉字的研究较少，尤其是在没有大量配对的汉字数据集的情况下。该模型使用内容和风格样式都不匹配的汉字数据集，将生成个性化手写汉字的过程公式化为一个从现有的标准印刷字体到个性化手写汉字样式映射的问题。在基于无监督学习的图像翻译模型的基础上，利用注意力机制和自适应标准化层来增强个性化汉字生成的内容和风格，并且通过改进损失函数提高了判别器网络的判别能力。在CASIA-HWDB手写汉字数据集和兰亭序书法数据集上进行了实验，通过对比内容准确性和风格差异性的评价指标，验证了该方法的有效性。

关键词: 生成对抗网络, 图像翻译, 无监督学习, 手写汉字生成

LU Peng, CHEN Jinyu, ZOU Guoliang, WAN Ying, ZHENG Zongsheng, WANG Zhenhua. Personalized Handwritten Chinese Character Generation Method for Unsupervised Image Translation[J]. Computer Engineering and Applications, 2022, 58(8): 221-229.

卢鹏, 陈金宇, 邹国良, 万莹, 郑宗生, 王振华. 无监督图像翻译的个性化手写汉字生成方法[J]. 计算机工程与应用, 2022, 58(8): 221-229.

References

[1] CHANG B，ZHANG Q，PAN S，et al.Generating handwritten Chinese characters using CycleGAN[C]//2018 IEEE Winter Conference on Applications of Computer Vision，Lake Tahoe，NV，2018：199-207.
[2] ZONG B，ZHU Y.Strokebank：automating personalized Chinese handwriting generation[C]//Proceedings of the National Conference on Artificial Intelligence，2014：3024-3029.
[3] PAN W，LIAN Z，SUN R，et al.Flexifont：a flexible system to generate personal font libraries[C]//Proceedings of the 2014 ACM Symposium on Document Engineering，New York，NY，USA，2014：17-20.
[4] LIAN Z，ZHAO B，CHEN X，et al.EasyFont：a style learning-based system to easily build your large-scale handwriting fonts[J].ACM Transactions on Graphics，2019，38：1-18.
[5] ZHANG X，YIN F，ZHANG Y，et al.Drawing and recognizing Chinese characters with recurrent neural network[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，40：849-862.
[6] TIAN Y，Rewrite：neural style transfer for Chinese fonts[EB/OL].[2020-06-23].https：//github.com/kaonashi-tyc/Rewrite.
[7] XIAO F，HUANG B，WU X.Automatic generation of Chinese handwriting via fonts style representation learning[J].arXiv：2004.03339，2020.
[8] RONNEBERGER O，FISCHER P，BROX T.U-net：convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention.Cham：Springer，2015：234-241.
[9] TIAN Y.zi2zi：master Chinese calligraphy with conditional adversarial networks[EB/OL].[2020-06-23].https：//github.com/kaonashi-tyc/zi2zi.
[10] ZHU J，PARK T，ISOLA P，et al.Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE International Conference on Computer Vision，Venice，2017：2242-2251.
[11] ISOLA P，ZHU J，ZHOU T，et al.Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，2017：5967-5976.
[12] XU F，MA B，CHANG H，et al.Style transfer with adversarial learning for cross-dataset person re-identification[C]//Computer Vision-ACCV.Cham：Springer，2018：165-180.
[13] RAFFIEE A，SOLLAMI M.GarmentGAN：photo-realistic adversarial fashion transfer[J/OL].arXiv：2003.01894，2020.
[14] REN Y，YU X，CHEN J，et al.Deep image spatial transformation for person image generation[J].arXiv：2003.
00696，2020.
[15] GOODFELLOW I J，POUGET-ABADIE J，MIRZA M，et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems，Cambridge，MA，USA，2014：2672-2680.
[16] HUANG H，YU P S，WANG C.An introduction to image synthesis with generative adversarial nets[J].arXiv：1803.
04469v2，2018.
[17] MIRZA M，OSINDERO S.Conditional generative adversarial nets[J].arXiv：1411.1784，2014.
[18] YI Z，ZHANG H，TAN P，et al.DualGAN：unsupervised dual learning for image-to-image translation[J].arXiv：1704.
02510，2017.
[19] KIM T，CHA M，KIM H，et al.Learning to discover cross-domain relations with generative adversarial networks[J].arXiv：1703.05192，2017.
[20] LIU M，BREUEL T，KAUTZ J.Unsupervised image-to-image translation networks[C]//Proceedings of the 31st Conference on Neural Information Processing Systems，2017：700-708.
[21] MNIH V，HEESS N，GRAVES A.Recurrent models of visual attention[C]//Neural Information Processing Systems，2014：2204-2212.
[22] XU K，BA J，KIROS R，et al.Show，attend and tell：neural image caption generation with visual attention[C]//Proceedings of the 32nd International Conference on Machine Learning，2015：2048-2057.
[23] FU J，ZHENG H，MEI T.Look closer to see better：recurrent attention convolutional neural network for fine-grained image recognition[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，2017：4476-4484.
[24] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Neural Information Processing Systems，2017：6000-6010.
[25] IOFFE S，SZEGEDY C.Batch normalization：accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning，2015：448-456.
[26] BA J L，KIROS J R，HINTON G E.Layer normalization[J].arXiv：1607.06450，2016.
[27] ULYANOV D，VEDALDI A，LEMPITSKY V.Instance normalization：the missing ingredient for fast stylization[J].arXiv：1607.08022v3，2016.
[28] WU Y，HE K M.Group normalization[J].arXiv：1803.
08494，2018.
[29] NAM H，KIM H E.Batch-instance normalization for adaptively style-invariant neural networks[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems，2018：2563-2572.
[30] ZHOU T，KRAHENBUHL P，AUBRY M，et al.Learning dense correspondence via 3D-guided cycle consistency[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，NV，2016：117-126.
[31] HE K M，ZHANG X Y，REN S Q，et al.Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition，Las Vegas，NV，2016：770-778.
[32] HU J，SHEN L，ALBANIE S，et al.Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，Salt Lake City，UT，2018：7132-7141.
[33] NAIR V，HINTON G E.Rectified linear units improve restricted Boltzmann machines[C]//Proceedings of the 27th International Conference on International Conference on Machine Learning，2010：807-814.
[34] MAO X D，LI Q，XIE H R，et al.Least squares generative adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision，Venice，Italy，2017.
[35] ALEXIA J M.The relativistic discriminator：a key element missing from standard GAN[J].arXiv：1807.00734v3，2018.
[36] LIU C，YIN F，WANG D H，et al.CASIA online and offline Chinese handwriting databases[C]//Proceedings of the 2013 12th International Conference on Document Analysis and Recognition，Beijing，China，2011：37-41.
[37] KINGMA D，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980v8，2014.
[38] GATYS L A，ECKER A S，BETHGE M.A neural algorithm of artistic style[J].arXiv：1508.06576v1，2015.