16-824 Group 18 Project Website

Prior Work

Prior approaches to the problem of rain removal included using convolutional neural networks (CNNs) to train on pairs of images with and without rain as well as using generative adversarial networks (GANs) in order to generate portions of the image without rain. In addition to model-centric methods of raindrop removal, research has been done focusing on the physical characteristics of raindrops to inspire a better understanding of the image area behind the raindrop [3]. There are two angles from research approaches deraining using the physical properties of raindrops. The first angle mostly models the raindrop itself while the second angle building end to end models based solely off of the photometric raindrop properties [3, 4].

Fig 3: The above model architecture demonstrates the second of the physics based deraining methods, computing the physical properties of the rain in the first part of the network and applying a GAN on the output of the physical component in order to completely derain the image. [4]

Baseline Work

The baseline for our model is a recurrent, generative, attentive model that processed input rainy images with attention and generated derained images as an output. The generator is composed of a recurrent network which serves as the attention map for the image, and the output from the recurrent network is processed through a CNN to derain the image. Then, another CNN is used in the discriminator to determine the loss on the reconstruction. [1]

Fig. 4: The network architecture of our baseline. The architecture is composed of a generative, recurrent, attentive component followed by CNN components for the output and disciminative processes. [1]