I have been reading the original Goodfellow, et. al. paper on Generative Adversarial Networks and the way that they can obtain estimates of the posterior distribution of a discriminative network or autoencoder. Now usually something that generates estimates from the posterior involves either an MCMC sampling scheme, or in some cases conjugacy of the prior.

So I was wondering if someone could clarify whether GANs can act as a replacement for MCMC sampling, or are there areas which only MCMC will work?

  • $\begingroup$ MCMC is a simulation method to approximate a Bayesian posterior distribution and if needed Bayes estimates. GANs are a form of (point) parameter estimation that can be produced by MCMC algorithms. It thus makes little sense to put MCMC and GANs at the same level. $\endgroup$ – Xi'an Dec 8 '18 at 18:50
  • $\begingroup$ @Xi'an thanks for the info. Yeah, this is what was confusing me. So if I have a bayesian model, I can use MCMC to obtain the posterior distribution and then get a point/parameter estimate. If I read your message correctly, a GAN just provides a point estimate. I am still not clear on the difference? Do you mean that GANs can provide a point estimate, but not the full distribution? Can you recommend a reference that lays out the guarantees of GANs or the mathematical details? The original Goodfellow paper is a bit thin on the actual math. $\endgroup$ – krishnab Dec 8 '18 at 19:03
  • $\begingroup$ You can check Saatchi's and Wilson's Bayesian GANs. $\endgroup$ – Xi'an Dec 9 '18 at 6:36
  • 1
    $\begingroup$ @Xi'an I'm a bit surprised that an expert of your standing would recommend that paper. The resulting model is conceptually flawed (incompatible conditionals) and the results are terrible (mode collapse all the way, low image quality, etc.). Unsurprisingly, even in a field which is not overly selective in terms of quality, that paper has received few citations, and excluding those from collaborators of the authors, most citations criticize their approach. $\endgroup$ – DeltaIV Dec 9 '18 at 13:55
  • 1
    $\begingroup$ @DeltaIV: Thanks, I have commented this paper, which, as you mentioned, is suffering from handling incompatible conditionals. Dan Roy (Toronto) also commented on that aspect earlier, when the paper was first arXived. This is however the only paper I have read on the topic. $\endgroup$ – Xi'an Dec 9 '18 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.