1
$\begingroup$

Instance Noise is a trick for stabilising Generative Adversarial Network (GAN) training. In this paper, the authors say that (page 14, fig. 6)

Instance Noise broadens the support of both distributions without biasing the optimal discriminator

I understand the intuition about how noise broadens the distributions and why this is beneficial for training. What I wonder is:

  1. Is there any formal demonstration of this? What is the hypothesis behind it?
  2. What happens if we use a different type of noise (e.g., not Gaussian)?

Thank you :)

Improve this question
$\endgroup$
3
  • $\begingroup$ I think you’re going to bias the optimal discriminator if you use noise that doesn’t have an expected value of 0. If you use noise with narrower support than you already have, I think you end up with the same support when you add the two distributions. Do you know about convolution of random variables. That’s going to come into play here. $\endgroup$
    – Dave
    Commented May 15, 2020 at 15:19
  • 1
    $\begingroup$ see here: en.wikipedia.org/wiki/Variance#Basic_properties -- note that if X and Y are independent, Var(X+Y) = Var(X) + Var(Y). $\endgroup$
    – Glen_b
    Commented May 16, 2020 at 8:34
  • $\begingroup$ Thanks, that was easy :) $\endgroup$
    – gab
    Commented May 18, 2020 at 10:53

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.