1
$\begingroup$

I am a bit confused about the terminology when using subscripts in expectations and probabilities. I was reading about the reparameterization trick from the following link:

"gregorygundersen - Link"
Where i stumbled upon the following:

$$ \begin{array}{l}{\boldsymbol{\epsilon} \sim p(\boldsymbol{\epsilon})} \\ {\mathbf{z}=g_{\boldsymbol{\theta}}(\boldsymbol{\epsilon}, \mathbf{x})}\end{array} $$ $$ \nabla_{\theta} \mathbb{E}_{p_{\theta}(\mathbf{z})}\left[f\left(\mathbf{z}^{(i)}\right)\right]=\nabla_{\theta} \mathbb{E}_{p(\epsilon)}\left[f\left(g_{\theta}\left(\epsilon, \mathbf{x}^{(i)}\right)\right)\right] $$

What do the subscripts of the expectations mean here? And also the subscripts of the probabilities, I can't really see how $$p_\theta(\mathbf{z})=p(\epsilon)$$ given the above, but I think it's because i don't fully understand the subscripts. Where can I find the terminology of these subscripts? Any links or useful books that introduce them?

Thanks for the help!

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

What do the subscripts of the expectations mean here?

They are the distribution you are taking the expectation with respect to. They are the "weights" you're using to calculate the weighted average.

I can't really see how...

This $$ \mathbb{E}_{p_{\theta}(\mathbf{z})}\left[f\left(\mathbf{z}^{(i)}\right)\right]= \mathbb{E}_{p(\epsilon)}\left[f\left(g_{\theta}\left(\epsilon, \mathbf{x}^{(i)}\right)\right)\right] $$ is called the law of the unconscious statistician (LOTUS). Notice in the second line how you're applying $g_{\theta}$ to epsilons. You can either take the expectation with respect to either density. You will get the same expected value.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.