# A technical question about the reparametrisation trick

I was reading this post which enlightened me about the technicalities of the reparametrisation trick, but I only get the intuition of this equivalent transform and I'm not sure why it is true: $$𝐸_𝑞[x^2]=𝐸_𝑝[(𝜃+𝜖)^2]$$

The intuition is clear - we have introduced another random variable which produces our previous random variable but our $$\epsilon$$ is sampled from a new random variable, which has different distribution.

The definitions are: $$q_{\theta}(x) = \mathcal{N}(\theta,1)$$ $$x = \theta + \epsilon$$ $$\epsilon \sim \mathcal{N}(0,1)$$

My attempt: I think p in this context means $$p(\epsilon) = \mathcal{N}(0,1)$$

So then the exercise is to prove:

$$E_q [ x^2] = \int_{X} x^2 q_{\theta}(x) dx = \int_{\eta} (\theta + \epsilon)^2 p(\epsilon) d\epsilon$$

I assume it is possible then to argue that for a given $$x_0 = \theta + \epsilon_0$$ the equation will only agree if we make a shift, but for me this is still too intuitive. Could we express this more rigorously? (feel free to throw Measure Theory at me if needed)

• $\epsilon$ is the random variable that follows a normal distirbution with mean $0$ and variance $1$. What do you mean by $p(\epsilon)$? – nbro Jun 13 at 15:32
• There's no exercise to be done, the equality holds above basically by definition since this is just a change of variables from x to $\epsilon$ en.wikipedia.org/wiki/… – aleshing Jun 13 at 15:51

## 1 Answer

This is merely a change of variable. The expectation operator uses the density of the random variable(s) inside the expression. However, if you just look from a calculus perspective: $$x=\theta+\epsilon\rightarrow dx=d\epsilon, \ \ \epsilon=x-\theta$$ $$q_\theta(x)=\frac{1}{\sqrt{2\pi}}\exp\left(\frac{-(x-\theta)^2}{2}\right)=\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{\epsilon^2}{2}\right)=p(\epsilon)$$ And, the limits of $$x$$, i.e. $$(-\infty,\infty)$$ are shifted by $$\theta$$, but, since infinity, they're the same. Or, you could leave them as in your notation, $$X\rightarrow\eta$$. Then, it follows that $$E_q[x^2]=E_p[(x+\theta)^2]$$

• Yes, I believe this is what i was looking at. The interpretation with calculus combined with the change of variable for probability densities makes me feel more comfortable about this. – boomkin Jun 14 at 10:09