1
$\begingroup$

I have been reading and looking at implementations of the SRGAN, from "Photo-realistic Single Image Super Resolution with Generative Adversarial Networks" paper. One thing that I noticed is that the SRGAN formulation does not consider a noise Z on it's input, unlike normal GANs/cGANs. In Super Resolution we want the model to learn to generate samples from a certain distribution (human faces) that are conditioned to a certain Low Resolution(LR) Image (must be that specific person), therefore isn't it a cGAN? But why we don't see any random noise as input on SRGAN? Would the G (LR image) be the G(z | y) of the cGAN?

The cGAN formulation follows: enter image description here

It is not clear to me that SRGAN uses the idea of cGANs, since we don't pass any random noise as input, only the LR image (deterministic, at least in the paper case).

The SRGAN formulation follows:

enter image description here

$\endgroup$
1
$\begingroup$

In some circumstances, noise is needed to prevent the discriminator from having a trivial job. For example if you want a conditional GAN to generate an MNIST digit conditioned on the label, then if you don't use noise you'll fail because the deterministic generator will only come up with 1 digit for each of the 10 labels, which the generator will easily pick out.

As the data gets richer, this becomes less of a problem. While you might expect a discriminator to notice that the generator can only produce one image for each digit, it probably doesn't have enough capacity to notice that your SR generator only generates one super-resolution image per low-resolution image -- that would require memorizing pretty much every low-res image in the training dataset, which would be quite difficult.

I'm sure there's also some probabilistic or game/information theoretic interpretation of (conditional) GANs under which removing the noise completely breaks the theory. However on a practical level it doesn't cause much problems when what you are conditioning on is sufficiently complex.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the answer! I have dug on this question and it seems that is indeed the case. At least the authors of pix2pix (which presents a similar situation, but for image-to-image translation) think the same. More info can be found on this github issue $\endgroup$ – Antonio Luis Sombra Apr 15 '19 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.