1
$\begingroup$

I've noticed in several resources on variational autoencoders (for example the Wikipedia article), we use the same parameters theta ($\theta$) for the prior, likelihood, posterior, etc distributions. For example the equation $p_\theta(x) = \int_z p_\theta(x|z)p_\theta(z)dz$. Aren't $p_\theta(x)$ and $p_\theta(z)$ two different distributions, so how can we parameterize them with the same $θ$ params. I might be misunderstanding something about what it means to parameterize a distribution with neural nets..

$\endgroup$

1 Answer 1

2
$\begingroup$

Of course, it is not the same parameters that are used to parametrize the likelihood or the posterior for example, but we keep the same letter (here $\theta$) to refer to the parameters of the true (original) distributions. On the other hand, in VAEs, $\phi$ is often used to refer to the parameters to the variational distributions. Indeedn, this can sometimes be unclear.

To address your second point, let us say $p_{\theta}(x|z)$ is a Gaussian distribution, classically it is parametrized by a scalar mean $m$ and scalar a variance $s^2$ : $p_{\theta}(x|z)=\mathcal{N}(x;m,s^2)$, here $\theta=\{m,s^2\}$. Now when we say that this distribution is parametrized by a neural network we mean that $m$ and $s^2$ will be the outputs of a neural network with input $z$. Let us denote this neural network $f_{\theta}$, $\theta$ is here the set of weights and biases of the neural network. We can write $p_{\theta}(x|z)=\mathcal{N}(x;f_{\theta}(z))$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.