0
$\begingroup$

I'm reading an excellent tutorial on variational autoencoders by Carl Doersch. However, he uses the following notation to define the generative distribution:

$$ P(X|z;\theta) = N(X|f(z;\theta), \sigma^{2} * I) $$

I interpret this to mean that our reconstruction of the observed variable $X$ is a normal distribution with $\mu$ defined by some function $f(z;\theta)$ and covariance $\sigma^{2}$.

And he uses this notation to describe the latent deep Gaussian distribution on $Z$:

$$P(Z) = N(Z|0,I)$$

Which I interpret to mean that

the latent variable Z is drawn from a (potentially multi-variate) normal distribution with $\mu$ vector of 0 and covariance $I$.

I don't understand what the difference between the above and simply writing

$$P(Z) = N(0,I)$$ and

$$P(X|z;\theta) = N(f(z; \theta), \sigma^{2} * I) $$.

What is the difference conceptually? Is there any, or is it just a matter of style?

$\endgroup$
2
$\begingroup$

That is the density notation in the paper. He reserves $\mathcal{N}(0,I)$ for referring to (normal) random variables (i.e. the distribution), and uses $\mathcal{N}(X|0,I)$ to refer to densities. For example, if $Z$ is distributed normally with zero-mean, identity covariance, we'd write $Z\sim\mathcal{N}(0,I)$, (as is in Page 11). When we're talking about $p_Z(z)$, that is something like $\mathcal{N}(z|0,I)$ in the paper. However, notice that Eq. 12, I believe mistakenly, doesn't follow this convention. This notation is also prevalent in the original VAE paper of Kingma.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.