关键词: 光场图像, 长短期记忆网络（LSTM）, 图像去雨, 图像恢复

Abstract:

Limited input information from single image will seriously affect the performance of rain streak removal. Compared with ordinary 2D images, Light Field Images（LFIs） can record abundant information of a 3D scene. Aiming at such problems and using light field image characteristics, a dual LSTM neural network based rain streaks removal method for LFIs is proposed. The proposed neural network includes two sub-networks. One is Rain Streaks Detecting Network（RSDNet） and the other one is Background Restoring Network （BRNet）. Proposed method contains three main steps. First, a matching-cost-optimization-based method is adopted to calculate the depth maps for sub-views within the 3D EPI. Second, RSDNet is used to extract rain streaks. Dual LSTM is proposed to transmit high frequency information of rain streaks to the BRNet. Third, the BRNet is used to restore the background LFI to obtain the rain-free sub-views. In order to train and evaluate the proposed neural network, a rainy light field dataset is constructed, in which each LFI is composed of a real-world LFI and a synthetic rain streak LFI. Extensive experimental results demonstrate that the proposed method can effectively restore rainy LFIs.

Key words: light field images, Long Short-Term Memory（LSTM）, rain removal, image restoration