结合线稿提示的热贡建筑彩绘图像多路编码上色网络

doi:10.3778/j.issn.1002-8331.2312-0414

摘要/Abstract

摘要： 建筑彩绘是人类非物质文化遗产热贡艺术的重要组成部分，一些年代久远存在褪色氧化现象的建筑彩绘因环境因素和传承现状的限制得不到及时的维护，这严重影响热贡建筑彩绘艺术的鉴赏和保护。针对现有的图像上色方法对建筑彩绘图像进行上色时存在的颜色混淆、串色漏色等问题，提出一种端到端的结合线稿提示的热贡建筑彩绘图像多路编码上色网络。该网络以提取的线稿图作为图像先验知识补充特征传递过程中忽视的纹理细节信息，加强网络对全局语义的理解，改善串色漏色的现象；多路并行编码模块以三个空洞卷积和传统卷积不同排列方式的编码分支为结构，通过获得不同尺度的的感受野以此更好地捕捉上下文信息，提高色彩特征的提取能力；结合空间注意力和通道注意力的注意力残差模块，通过对特征信息的重新整合加工，帮助网络全面理解图像色彩特征和纹理信息的空间分布，提高上色质量。所提方法在构建的热贡建筑彩绘数据集和动漫数据集上进行对比实验，与现有上色方法相比，所提方法在定性分析和定量分析中具有较好的表现，在主观评价的对比中也获得与人类主观审美和专业要求最一致的结果。实验证明，所提方法图像上色效果优于其他上色方法，在热贡建筑彩绘图像上色领域具有较大应用价值。

关键词: 热贡建筑彩绘, 图像上色, 多路并行编码, 注意力残差, 线稿提示

Abstract: Architectural colored drawing is an important component of the human intangible cultural heritage of Regong art. Some ancient Regong architectural colored drawings have the phenomenon of fading and oxidation, which cannot be maintained promptly due to environmental factors and the limitations of inheritance status. This seriously affects the appreciation and protection of the Regong architectural colored drawings art. Aiming at the problems of color confusion and color leakage when using existing image coloring methods to color architectural colored drawing images, this thesis proposes an end-to-end multiple encoding coloring network for Regong architectural colored drawing images that combines sketch hints. This network supplements the ignored texture details in the feature transfer process with extracted sketch as prior knowledge of the image, enhances the network’s understanding of global semantics, and improves the phenomenon of color distortion and leakage. The multipath parallel encoding module is structured with three encoding branches arranged in different ways using dilated convolutions and traditional convolutions. By obtaining receptive fields of various scales, it better captures contextual information and improves the ability to extract color features. The attention residual module, which combines spatial attention and channel attention, helps the network comprehensively understand the spatial distribution of image color and texture information through the fusion processing of feature information, thereby improving the quality of coloring. Finally, a comparative experiment is conducted on the constructed Regong architectural colored drawing dataset and animation dataset using the proposed method. Compared with existing coloring methods, the proposed method performs well in qualitative and quantitative analysis and achieves the most consistent results with human subjective aesthetics and professional requirements in subjective evaluation comparison. Experimental results have shown that the proposed method has better image coloring effects than other coloring methods and has significant application value in the field of the coloring of Regong architectural colored drawings art.

Key words: Regong architectural colored drawing, image coloring, multipath parallel encoding, attention residual, sketch hints

程苗, 张效娟, 赵洋, 范虹. 结合线稿提示的热贡建筑彩绘图像多路编码上色网络[J]. 计算机工程与应用, 2025, 61(9): 277-287.

CHENG Miao, ZHANG Xiaojuan, ZHAO Yang, FAN Hong. Multiple Encoding Coloring Network for Regong Architectural Colored Drawing Images Combining Sketch Hints[J]. Computer Engineering and Applications, 2025, 61(9): 277-287.

参考文献

[1] 王晓珍. 甘青河湟地区藏汉古建筑彩画研究[M]. 北京:中国文联出版社, 2016: 215-216.
WANG X Z. Research of architectural decorative paintings of Hehuang River basin in Gansu and Qinghai Province[M]. Beijing: China Federation of Literary and Art Circles Press, 2016: 215-216.
[2] 董建民, 李丽宁. 基于色彩向量的藏式传统建筑外观数字特征提取[J]. 软件导刊, 2018, 17(7): 219-222.
DONG J M, LI L N. Feature extraction on appearance of Tibetan traditional architecture based on color vectors[J]. Software Guide, 2018,17(7): 219-222.
[3] QU Y, WONG T T, HENG P A. Manga colorization[J]. ACM Transactions on Graphics (ToG), 2006, 25(3): 1214-1220.
[4] ZHANG S, ZHOU J. Attention-based unsupervised sketch colorization of anime avatar[C]//Proceedings of the 2022 2nd International Conference on Control and Intelligent Robotics, 2022: 829-833.
[5] CHEN Y, LUO Y, DING Y, et al. Automatic colorization of images from Chinese black and white films based on CNN[C]//Proceedings of the 2018 International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, China, 2018: 97-102.
[6] 呼伟, 徐巧枝, 葛湘巍, 等. 医学图像分割的无监督域适应研究综述[J]. 计算机工程与应用, 2024, 60(6): 10-26.
HU W, XU Q Z, GE X W, et al. Review of unsupervised domain adaptation in medical image segmentation[J]. Computer Engineering and Applications, 2024, 60(6): 10-26.
[7] LEVIN A, LISCHINSKI D, WEISS Y. Colorization using optimization[J]. ACM Transactions on Graphics, 2004, 23(3): 689-694.
[8] WELSH T, ASHIKHMIN M, MUELLER K. Transferring color to greyscale images[C]//Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, 2002: 277-280.
[9] DESHPANDE A, ROCK J, FORSYTH D. Learning large-scale automatic image colorization[C]//Proceedings of the IEEE International Conference on Computer Vision, 2015: 567-575.
[10] VITORIA P, RAAD L, BALLESTER C. ChromaGAN: adversarial picture colorization with semantic class distribution[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2020: 2445-2454.
[11] WU Y, WANG X, LI Y, et al. Towards vivid and diverse image colorization with generative color prior[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 14377-14386.
[12] JI X, JIANG B, LUO D, et al. ColorFormer: image colorization via color memory assisted hybrid-attention transformer[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 20-36.
[13] 赵雪雅, 彭春燕, 张效娟. 基于自适应多特征融合的双重图像退化修复网络[J]. 中国科学:技术科学, 2023, 53(11):1939-1952.
ZHAO X Y, PENG C Y, ZHANG X J. Dual image degradation repair network based on adaptive multi-feature fusion[J]. Scientia Sinica(Technologica), 2023, 53(11):1939-1952.
[14] LU P, YU J, PENG X, et al. Gray2colornet: transfer more colors from reference image[C]//Proceedings of the 28th ACM International Conference on Multimedia, 2020: 3210-3218.
[15] LI Z, GENG Z, KANG Z, et al. Eliminating gradient conflict in reference-based line-art colorization[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 579-596.
[16] ZHANG L, LI C, WONG T T, et al. Two-stage sketch colorization[J]. ACM Transactions on Graphics (TOG), 2018, 37(6): 1-14.
[17] ZHANG L M. Style2paints V4. 5[EB/OL]. (2022-06-15)[2023-08-02]. https://github.com/lllyasviel/style2paints/tree/master/V4. 5.
[18] ZHANG L, LI C, SIMO-SERRA E, et al. User-guided line art flat filling with split filling mechanism[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021: 9889-9898.
[19] JIANG Y, GONG X, LIU D, et al. Enlightengan: deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing, 2021, 30: 2340-2349.
[20] WINNEM?LLER H, KYPRIANIDIS J E, OLSEN S C. XDoG: an extended difference-of-gaussians compendium including advanced image stylization[J]. Computers & Graphics, 2012, 36(6): 740-753.
[21] 赵磊, 吉柏言, 邢卫, 等. 基于多路编码器和双重注意力的古画修复算法[J]. 计算机研究与发展, 2023, 60(12): 2814-2831.
ZHAO L, JI B Y, XING W, et al. Ancient painting inpainting algorithm based on multi-channel encoder and dual attention[J]. Journal of Computer Research and Development, 2023,60(12): 2814-2831.
[22] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[J]. arXiv:1511.07122, 2015.
[23] HE M, CHEN D, LIAO J, et al. Deep exemplar-based colorization[J]. ACM Transactions on Graphics (TOG), 2018, 37(4): 1-16.
[24] HEUSEL M, RAMSAUER H, UNTERTHINER T, et al. GANs trained by a two time-scale update rule converge to a local nash equilibrium[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6629-6640.
[25] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 586-595.
[26] HASLER D, SUESSTRUNK S E. Measuring colorfulness in natural images[C]//Proceedings of the Human Vision and Electronic Imaging VIII, 2003: 87-95.
[27] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1125-1134.