关键词: 深度学习, 图像修复, 注意力机制, 特征调整

Abstract: Deep learning has brought unprecedented advances to image inpainting. However, existing methods often produce structurally distorted, texture-blurred image inpainting results after feature merging because of the spatial deviation between the upsampled feature maps and the corresponding bottom-up feature maps due to the repeated use of downsampling operations in the feature extraction process. To address these problems, the paper proposes an image inpainting model based on contextual feature adjustment and joint attention. The model consists of two parts：the context feature adjustment module and the Joint self-attention module. The context feature adjustment module reduces the spatial deviation by adjusting each sampling position in the convolution kernel and learning the transformation offset of pixels for aligning the up-sampled features in the context. The joint self-attention module can effectively model the long-distance dependence between input and output features by maintaining a relatively high resolution in the space and channel dimensions and adopting the nonlinear function of the Softmax-Sigmoid joint so that the model can achieve better performance in image inpainting tasks. The integration of these two modules into a top-down pyramid structure enhances the model’s use of different scale features of the image and forms a new image inpainting model. The proposed method is evaluated on publicly available datasets such as CelebA, Places2, and Paris StreetView. Experimental results show that the proposed method outperforms the current mainstream image inpainting techniques in terms of quality and quantity.

Key words: deep learning, image inpainting, attention mechanism, feature adjustment