Clarification: Are Generative Adversarial Networks an alternative to MCMC sampling? I have been reading the original Goodfellow, et. al. paper on Generative Adversarial Networks and the way that they can obtain estimates of the posterior distribution of a discriminative network or autoencoder. Now usually something that generates estimates from the posterior involves either an MCMC sampling scheme, or in some cases conjugacy of the prior. 
So I was wondering if someone could clarify whether GANs can act as a replacement for MCMC sampling, or are there areas which only MCMC will work?
 A: Both Markov Chain Monte Carlo (MCMC) and the generator network from a Generative Adversarial Network (GAN) return samples from a probability distribution.
However, they solve different problems:
MCMC works when known the formula for the probability of each configuration (it does not need to be normalized). A classic example is an Ising model, in which the probability of a configuration is $\exp(- \beta E)$, where $\beta$ is inverse temperature. To sample, we can use the Metropolis-Hasting MCMC algorithm - flipping a single spin, with probability related to energies of the states. MCMC is often being used for integration in Bayesian statistics. In principle (given infinite time) we always get the exact numerical result.
In GANs we do not know the probability distribution. Instead, we know a few samples from which we want to create a sampler from this probability distribution. We don't get a function that says what is the probability of a given sample (at least, not with typical discriminator network that does only guess if the sample is real or generated).
Furthermore, in the case of GANs the problem is fuzzy - the function depends on the neural network architecture, training process, and other factors. In general, there are infinitely many ways to turn a discrete set of samples into a probability distribution (sampler).
GANs and MCMC are not exclusive. As already pointed out by @shimao, you can combine both of the approaches as in Metropolis-Hastings Generative Adversarial Networks (2018).
Also, there is an overlap of functionality, for which either can be used. For example, patterns can be generated either with GANs or with MCMC (here are some beautiful examples with ConvChain). In the later, we typically it is more reliable and less data-hungry, but we need to translate input into probability (based on some assumptions).
