Computer Engineering and Applications ›› 2021, Vol. 57 ›› Issue (22): 241-246.DOI: 10.3778/j.issn.1002-8331.2007-0279

• Graphics and Image Processing • Previous Articles     Next Articles

Research on Image Restoration Method Based on Structure Embedding

WANG Haiyong, LI Haiyang, GAO Xuejiao   

  1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
  • Online:2021-11-15 Published:2021-11-16

基于结构嵌入的图像修复方法研究

王海涌,李海洋,高雪娇   

  1. 兰州交通大学 电子与信息工程学院,兰州 730070

Abstract:

In view of the current problem in the field of image repair, there is a problem of lost structure and blurred texture, and it is not possible to make full use of background information to generate a filled area with a consistent content style. Based on the encoder-decoder network, this paper proposes a shared repair model with multi-scale structure information and attention mechanism. In the generation stage, multi-scale structure information is embedded to provide prerequisites for image restoration. At the same time, the multi-scale attention mechanism is used to obtain relevant information on the background information, and refine the content and structure related to the image. This model uses PatchGAN and fixed-weight VGG-16 classifier as the discriminator, and uses style loss and perception loss to the adversarial network in order to achieve the style consistency of the generated images. Compared with the current  mainstream image repair algorithms on the Places2 dataset, the results show that proposed the algorithm can restore the detailed information about the image structure better than other algorithms, and generate clearer and more detailed repair results.

Key words: Generative Adversarial Networks(GAN), attention mechanism, VGG-16, image repair

摘要:

针对当前图像修复领域存在结构丢失、纹理模糊、不能够充分利用背景信息生成内容风格一致的填充区域的问题,在编码解码网络基础上,提出带有多尺度结构信息与注意力机制的共享修复模型。在生成阶段,嵌入多尺度结构信息为图像修复提供前提条件。同时使用多尺度注意力机制,从背景信息中获取相关信息,并经过细化,生成与图像相关的内容和结构;使用PatchGAN和固定权重VGG-16分类器作为鉴别器,并将风格损失和感知损失引入到对抗网络中,以实现所生成图像的风格一致性。在Places2数据集上与当前主流的图像修复算法进行对比,实验结果表明该算法与其他算法相比能较好地恢复图像结构的细节信息,生成更清晰、精细的修复结果。

关键词: 生成对抗网络(GAN), 注意力机制, VGG-16, 图像修复