Computer Engineering and Applications ›› 2022, Vol. 58 ›› Issue (16): 257-264.DOI: 10.3778/j.issn.1002-8331.2101-0178

• Graphics and Image Processing • Previous Articles     Next Articles

Parallel Decoder Image Inpainting Method for Edge-Terminal Collaboration

HUO Xiangzuo, ZHANG Wendong, TIAN Shengwei, HOU Shuxiang   

  1. 1.School of Software, Xinjiang University, Urumqi 830008, China
    2.Key Laboratory of Software Engineering Technology, Xinjiang University, Urumqi 830008, China
  • Online:2022-08-15 Published:2022-08-15



  1. 1.新疆大学 软件学院,乌鲁木齐 830008
    2.新疆大学 软件工程技术重点实验室,乌鲁木齐 830008

Abstract: Focusing on the problems of poor effect, long response time and high energy consumption of existing neural network image inpainting methods for mobile terminal equipment, this paper puts forward an edge-terminal collaboration-oriented image inpainting method for parallel decoder and calculating offloading strategy. Firstly, based on the edge-terminal collaboration characteristics of mobile edge computing(MEC) technology, the edge-terminal collaboration-oriented, gated convolutional image inpainting network of parallel decoder, ETG-Net(edge-terminal gated convolution network), is proposed. Secondly, through the edge-terminal sharing of weights, the efficiency of image inpainting and training is improved, and the independent working ability of the mobile terminals is retained. Finally, based on the calculating offloading strategy, some computing tasks of image inpainting will be selectively offloaded to the edge cloud, so as to further reduce the calculation delay and energy consumption of the terminal nodes. Experimental results show that, compared with the advanced models in recent years, the proposed model not only ensures the quality of image inpainting, but also solves the problems of deploying image inpainting models on mobile terminal devices, and reduces the time delay of task response.

Key words: image inpainting, deep learning, generative adversarial network, mobile edge computing, gated convolution, weight sharing

摘要: 针对现有神经网络图像修复方法在移动终端设备上部署存在效果差、响应时间长、高能耗的问题,提出了一种面向边-端协同的并行解码器图像修复方法及计算卸载策略。结合移动边缘计算(mobile edge computing,MEC)技术边-端协同的特性,提出一种面向边-端协同的并行解码器门控卷积图像修复网络ETG-Net(edge-terminal gated convolution network)。通过边-端共享权值的方式,提升图像修复及训练效率,并保留移动终端的独立工作能力。基于计算卸载决策,将图像修复部分计算任务有选择地卸载至边缘云,进一步降低终端节点的计算时延和能耗。实验结果表明,与近年来先进的模型相比,所提模型在保证图像修复质量的同时,解决了移动终端设备上部署图像修复模型存在的问题,降低了任务的响应时延。

关键词: 图像修复, 深度学习, 生成对抗网络, 移动边缘计算, 门控卷积, 权值共享