Recently I have learned about Generative Adversarial Networks.
For training the Generator, I am somehow confused how it learns. Here is an implemenation of GANs:
`# train generator z = Variable(xp.random.uniform(-1, 1, (batchsize, nz), dtype=np.float32)) x = gen(z) yl = dis(x) L_gen = F.softmax_cross_entropy(yl, Variable(xp.zeros(batchsize, dtype=np.int32))) L_dis = F.softmax_cross_entropy(yl, Variable(xp.ones(batchsize, dtype=np.int32))) # train discriminator x2 = Variable(cuda.to_gpu(x2)) yl2 = dis(x2) L_dis += F.softmax_cross_entropy(yl2, Variable(xp.zeros(batchsize, dtype=np.int32))) #print "forward done" o_gen.zero_grads() L_gen.backward() o_gen.update() o_dis.zero_grads() L_dis.backward() o_dis.update()`
So it computes a loss for the Generator as it is mentioned in the paper. However, it calls the Generator backward function based on the Discriminator output. The discriminator output is just a number (not an array).
But we know that in general, for training a network, we compute a loss function in the last layer (a loss between the last layers output and the real output) and then we compute the gradients. So for example, if the output is 64*64, then we compare it with a 64*64 image and then compute the loss and do the back propagation.
However, in the codes that I see in Generative Adversarial Networks, I see they compute a loss for the Generator from the discriminator output (which is just a number) and then they call the back propagation for Generator. The Generators last layers is for example 64*64 pixels but the discriminator loss is 1*1 (which is different from the usual networks) So I do not understand how it cause the Generator to be learned and trained?
I thought if we attach the two networks (attaching the Generator and Discriminator) and then call the back propagation but just update the Generators parameters, it makes sense and it should work. But what I see in the codes are totally different.
So I am asking how it is possible?