Improved Training of Generative Adversarial Networks Using Representative Features

People

Duhyeon Bang and Hyunjung Shim

Abstract

Despite of the success of Generative Adversarial Networks (GANs) for image generation tasks, the trade-off between image diversity and visual quality are an well-known issue. Conventional techniques achieve either visual quality or image diversity; the improvement in one side is often the result of sacrificing the degradation in the other side. In this paper, we aim to achieve both simultaneously by improving the stability of training GANs. A key idea of the proposed approach is to implicitly regularizing the discriminator using a representative feature. For that, this representative feature is extracted from the data distribution, and then transferred to the discriminator for enforcing slow updates of the gradient. Consequently, the entire training process is stabilized because the learning curve of discriminator varies slowly. Based on extensive evaluation, we demonstrate that our approach improves the visual quality and diversity of state-of-the art GANs.

Overview and Contributions

1. We employ additional representative features, h_1, extracted from a pre-trained AE for implicitly constraining the discriminator’s update. In this way, the learning curve of GANs varies slowly, thus the GAN training is stabilized. As a result, we simultaneously achieve the visual quality and image diversity of GANs in an unsupervised manner.

2. Our framework of combining the pre-trained AE is easily extendable to various GANs using different divergences or structures. Also, the proposed model is robust against the parameter selections; all results in this paper use the same hyper-parameters suggested by a baseline GAN.

3. We conduct extensive experimental evaluations to show the effectiveness of RFGAN; our approach improves existing GANs including GANs with the gradient penalty.

Results

1. Toy example w.r.t mode collapse

To evaluate how well the proposed model could achieve the diversity of data generation (i.e. solving mode collapse), we trained our network with a simple 2D mixture of 8 Gaussians suggested by (Metz et al., 2016).

2. Qualitative evaluation

We compared the generated images from DCGAN and RFGAN from the same training iteration and found that our RFGAN clearly produced better results.

3. Interpolation

We reused the training data for extracting the representative features, it is possible to question if the gain came from overfitting of the training data. To justify that our achievement is not the result of data overfitting, we generated samples by walking in latent space. According to (Radford et al., 2015), (Bengio et al., 2013), and (Dinh et al., 2016), the interpolated images between two images in latent space do not have meaningful connectivity (i.e. there is a lack of smooth transitions)

Publication

Improved Training of Generative Adversarial Networks Using Representative Features,

D Bang, H Shim, International Conference on Machine Learning (ICML), 2018, July.

URLs

Link of paper and more information including quantitative evaluation.

[arXiv][IEEE Xplore(TBA)]

Reference

Metz, Luke, Poole, Ben, Pfau, David, and Sohl-Dickstein, Jascha. Unrolled generative adversarial networks. arXiv preprint arXiv:1611.02163, 2016.

Radford, Alec, Metz, Luke, and Chintala, Soumith. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434, 2015.

Bengio, Yoshua, Mesnil, Gregoire, Dauphin, Yann, and ´Rifai, Salah. Better mixing via deep representations. In Proceedings of the 30th International Conference on Machine Learning (ICML-13), pp. 552–560, 2013.

Dinh, Laurent, Sohl-Dickstein, Jascha, and Bengio, Samy. Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016.