I just started reading about GAN theory properly for the first time and I have a question about a comment in the original GAN paper.

On page two there's a paragraph that states the following:

... had developed more general stochastic backpropagation rules, allowing one to backpropagate through Gaussian distributions with finite variance, and to backpropagate to the covariance parameter as well as the mean.

Can anyone explain what backpropagating through a distribution means? In other words, what's the concrete meaning of this?


The quoted paragraph (actually appearing in the NIPS version of GAN paper) is dealing with variational autoencoders (VAE). Particularly, it refers to their reparametrization trick, which allows efficient computation of gradients of expected values over distributions parametrized by the differentiated parameters.

How exactly this reparametrization works is described nicely in: How does the reparameterization trick for VAEs work and why is it important?

Another great, easy to understand explanation is in the blog post The reparametrization trick by Gregory Gundersen.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.