MCFA-UNet：结合多尺度融合与注意力机制的图像生成网络

doi:10.3778/j.issn.1002-8331.2405-0306

摘要/Abstract

摘要： 在图像生成领域，基于去噪扩散概率模型（DDPM）的深度学习方法已经取得了显著的进展。然而，在处理复杂纹理和细节丰富的图像时，现有模型生成的图像会出现模糊、纹理细节不清晰等问题，主要原因是原始DDPM采用的UNet网络在捕捉高度细节化图像特征时存在一定局限性。为解决这一问题，提出了一种基于多尺度卷积和融合注意力机制的新型UNet网络，命名为MCFA-UNet。该网络通过在编码器和解码器中引入残差块和线性注意力多尺度卷积模块，并在跳跃连接中加入多尺度融合注意力组件，提升了对图像细节的捕捉能力及生成图像的整体质量。实验结果显示，在唐卡数据集、Cifar10和ImageNet-64公共数据集上，采用MCFA-UNet的DDPM模型优于原始的DDPM模型，得到了更低的FID值和更高的主观评价得分，证明了其改进效果的显著性。

关键词: 图像生成, 去噪扩散概率模型（DDPM）, UNet网络, AIGC方法

Abstract: In the field of image generation, deep learning methods based on denoising diffusion probabilistic models (DDPM) have made significant progress. However, when dealing with complex textures and detail-rich images, the images generated by existing models suffer from blurring and unclear texture details, mainly due to the limitations of the UNet network used in the original DDPM in capturing highly detailed image features. To solve this problem, a novel UNet network based on multi-scale convolution and fused attention mechanism, named MCFA-UNet, is proposed. This network improves the ability to capture image details and the overall quality of the generated images by introducing residual block and linear attention multi-scale convolution modules in the encoder and decoder, and adding a multi-scale fused attention component in the jump connection. The experimental results show that the DDPM model with MCFA-UNet outperforms the original DDPM model on the Tangka dataset, Cifar10 and ImageNet-64 public datasets, obtaining lower FID values and higher subjective evaluation scores, which proves the significance of the improvement.

Key words: image generation, denoising diffusion probabilistic models (DDPM), UNet network, artificial intelligence generated content (AIGC)

王铁君, 张泽宇, 郭晓然, 武娇. MCFA-UNet：结合多尺度融合与注意力机制的图像生成网络[J]. 计算机工程与应用, 2025, 61(12): 222-231.

WANG Tiejun, ZHANG Zeyu, GUO Xiaoran, WU Jiao. MCFA-UNet: Combining Multiscale Fusion and Attention for Image Generation Networks[J]. Computer Engineering and Applications, 2025, 61(12): 222-231.

参考文献

[1] 张泽宇, 王铁君, 郭晓然, 等. AI绘画研究综述[J]. 计算机科学与探索, 2024, 18(6): 1404-1420.
ZHANG Z Y, WANG T J, GUO X R, et al. Survey of AI painting[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(6): 1404-1420.
[2] OPENAI. GPT-4 technical report[J]. arXiv:2303.08774, 2023.
[3] 陈吉尚, 哈里旦木·阿布都克里木, 梁蕴泽, 等. 深度学习在符号音乐生成中的应用研究综述[J]. 计算机工程与应用, 2023, 59(9): 27-45.
CHEN J S, ABUDUKELIMU H, LIANG Y Z, et al. Review of application of deep learning in symbolic music generation[J]. Computer Engineering and Applications, 2023, 59(9): 27-45.
[4] KINGMA D P, WELLING M. Auto-encoding variational Bayes[J]. arXiv:1312.6114, 2013.
[5] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Advances in Neural Information Processing Systems, 2014: 2672-2680.
[6] RAMESH A, DHARIWAL P, NICHOL A, et al. Hierarchical text-conditional image generation with CLIP latents[J]. arXiv:2204.06125, 2022.
[7] RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2015: 234-241.
[8] 徐光宪, 冯春, 马飞. 基于UNet的医学图像分割综述[J]. 计算机科学与探索, 2023, 17(8): 1776-1792.
XU G X, FENG C, MA F. Review of medical image segmentation based on UNet[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(8): 1776-1792.
[9] HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[J]. arXiv:2006.11239, 2020.
[10] NICHOL A Q, DHARIWAL P. Improved de-noising diffusion probabilistic models[C]//Proceedings of the International Conference on Machine Learning, 2021: 8162-8171.
[11] SONG J M, MENG C L, ERMON S. Denoising diffusion implicit models[J]. arXiv:2010.02502, 2020.
[12] WANG Z D, JIANG Y F, ZHENG H J, et al. Patch diffusion: faster and more data-efficient training of diffusion models[C]//Advances in Neural Information Processing Systems, 2024.
[13] DIAO Z H, GUO P L, ZHANG B H, et al. Maize crop row recognition algorithm based on improved UNet network[J]. Computers and Electronics in Agriculture, 2023, 210: 107940.
[14] ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 10674-10685.
[15] ZHOU Z W, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[M]//Deep learning in medical image analysis and multimodal learning for clinical decision support. Cham: Springer International Publishing, 2018: 3-11.
[16] ZHANG J W, ZHANG Y C, JIN Y Z, et al. MDU-Net: multi-scale densely connected U-Net for biomedical image segmentation[J]. Health Information Science and Systems, 2023, 11(1): 13.
[17] QIN X B, ZHANG Z C, HUANG C Y, et al. U2-Net: going deeper with nested U-structure for salient object detection[J]. Pattern Recognition, 2020, 106: 107404.
[18] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems, 2017.
[19] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[20] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[21] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[22] RAMACHANDRAN P, ZOPH B, LE Q V. Searching for activation functions[J]. arXiv:1710.05941, 2017.
[23] OUYANG D L, HE S, ZHANG G Z, et al. Efficient multi-scale attention module with cross-spatial learning[C]//Proceedings of the ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2023: 1-5.
[24] CHEN Z X, HE Z W, LU Z M. DEA-net: single image dehazing based on detail-enhanced convolution and content-guided attention[J]. IEEE Transactions on Image Processing, 2024, 33: 1002-1015.
[25] ZHANG X Y, ZHOU X Y, LIN M X, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 6848-6856.
[26] HEUSEL M, RAMSAUER H, UNTERTHINER T, et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017: 6629-6640.
[27] GONG X Y, CHANG S Y, JIANG Y F, et al. AutoGAN: neural architecture search for generative adversarial networks[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 3223-3233.