How is the generator in a GAN trained?

Question

The paper on GANs says the discriminator uses the following gradient to train:

$$\nabla _{\theta_d} \frac{1}{m}\sum^{m}_{i=1} [\log{D(x^{(i)})} + \log{(1-D(G(z^{(i)})))}]$$

The $z$ values are sampled, passed through the generator to generate data samples, and then the discriminator is backpropogated using the generated data samples. Once the generator generates the data, it plays no further role in the training of the discriminator. In other words, the generator can be completely removed from the metric by having it generate data samples and then only working with the samples.

I'm a bit more confused about how the generator is trained though. It uses the following gradient:

$$\nabla _{\theta_g} \frac{1}{m}\sum^{m}_{i=1} [\log{(1-D(G(z^{(i)})))}]$$

In this case, the discriminator is part of the metric. It cannot be removed like the previous case. Things like least squares or log likelihood in regular discriminative models can easily be differentiated because they have a nice, close formed definition. However, I'm a bit confused about how you backpropogate when the metric depends on another neural network. Do you essentially attach the generator's outputs to the discriminator's inputs and then treat the entire thing like one giant network where the weights in the discriminator portion are constant?

Answer 1

It helps to think of this process in pseudocode. Let generator(z) be a function that takes a uniformly sampled noise vector z and returns a vector of same size as input vector X; let's call this length d. Let discriminator(x) be a function that takes a d dimensional vector and returns a scalar probability that x belongs to true data distribution. For training:

G_sample = generator(Z)
D_real = discriminator(X)
D_fake = discriminator(G_sample)

D_loss = maximize mean of (log(D_real) + log(1 - D_fake))
G_loss = maximize mean of log(D_fake)

# Only update D(X)'s parameters
D_solver = Optimizer().minimize(D_loss, theta_D)
# Only update G(X)'s parameters
G_solver = Optimizer().minimize(G_loss, theta_G)

# theta_D and theta_G are the weights and biases of D and G respectively
Repeat the above for a number of epochs

So, yes, you are right that we essentially think of the generator and discriminator as one giant network for alternating minibatches as we use fake data. The generator's loss function takes care of the gradients for this half. If you think of this network training in isolation, then it is trained just as you would usually train a MLP with its input being the last layer's output of the generator network.

You can follow a detailed explanation with code in Tensorflow here(among many places): http://wiseodd.github.io/techblog/2016/09/17/gan-tensorflow/

It should be easy to follow once you look at the code.

Answer 2

Do you essentially attach the generator's outputs to the discriminator's inputs ?>and then treat the entire thing like one giant network where the weights in the >discriminator portion are constant?

Shortly: Yes.(I dug some of the GAN's sources to double check this)

There are also a lot of more into GAN training like: should we update D and G every time or D on odd iterations and G on even, and a lot more. There is also a very nice paper about this topic:

"Improved Techniques for Training GANs"

Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen

https://arxiv.org/abs/1606.03498

Answer 3

I have found this great resource: https://developers.google.com/machine-learning/gan/training

This is part of what came in it So we train the generator with the following procedure: 1- Sample random noise. 2- Produce generator output from sampled random noise. 3- Get discriminator "Real" or "Fake" classification for generator output. 4- Calculate loss from discriminator classification. 5- Backpropagate through both the discriminator and generator to obtain gradients. 6- Use gradients to change only the generator weights.

Answer 4

Recently I have uploaded collection of various GAN models on github repo. It is torch7 based, and very easy to run. The code is simple enough to understand with experimental results. Hope this will help

https://github.com/nashory/gans-collection.torch