Expectation with respect to a transformed random variable

Question

Problem

Suppose I have a random variable $z$ following a distribution $p(z)$. Suppose I have a transformation

$$ f(z) = x $$

that transforms the random variable $z$ into a new random variable $x$ with distribution $p(x)$. I have often seen the following result

$$ \mathbb{E}_{p(x)}[g(x)] = \mathbb{E}_{p(z)}[g(f(z))]. $$

In other words, the expectation with respect to a distribution $p(x)$ can be written in terms of the original distribution $p(z)$.

Is there proof of this? I think this should work even if $f(z)$ is not invertible and/or differentiable.

My Set-Up for a Solution

I will describe my measure theory set up.

Distribution of Z

Suppose we have two measurable spaces $(\Omega, \mathcal{F})$ and $(\mathsf{Z}, \mathcal{Z})$. The random variable $Z$ is a measurable mapping $$ Z: \Omega \to \mathsf{Z} $$

such that the pre-image $Z^{-1}(B)$ of any $\mathcal{Z}$-measurable set $B\in \mathcal{Z}$ is also $\mathcal{F}$-measurable:

$$ Z^{-1}(B) = \{\omega\in \Omega \, :\, Z(\omega) \in B \} \in \mathcal{F} \qquad \forall \, B \in \mathcal{Z} $$ Now the distribution of $Z$ is a push-forward measure. Suppose we have a probability space $(\Omega, \mathcal{F}, \mathbb{P})$. This means we can measure sets in $\mathcal{F}$. The push-forward or distribution for $Z$ is a way of measuring sets in $\mathcal{Z}$ via $\mathbb{P}$.

Basically, the distribution $Z_*\mathbb{P}$ assigns to sets $B\in\mathbb{Z}$ the same value as if we map $B$ back to $\mathcal{F}$ via $Z^{-1}$ first, and then we find its measure using $\mathbb{P}$.

$$ (\mathbb{P} \circ Z^{-1})(B) = Z_*\mathbb{P}(B) \qquad \forall \, B\in \mathcal{Z} $$

Distribution of X

Now, the new random variable $X$ is basically a function of the random variable $Z$ and therefore $X$ is also a random variable.

Consider the probability space $(\mathsf{Z}, \mathcal{Z}, Z_*\mathbb{P})$ for $Z$. Consider also a measurable function $$ X: \mathsf{Z} \to \mathsf{X} $$ where $(\mathsf{X}, \mathcal{X})$ is a measurable space. This essentially defines the random variable $X$. Since we can measure sets in $\mathcal{Z}$ using $Z_*\mathbb{P}$ we would like to measure sets in $\mathcal{X}$ too. To do this, we define the distribution of $X$ to be a push-forward measure. Essentially, to give a measure to a set $C\in \mathcal{X}$ it maps it to $\mathcal{Z}$ via $X^{-1}$ and then measures it with the distribution $Z_*\mathbb{P}$.

$$ (Z_*\mathbb{P} \circ X^{-1})(C) = X_*Z_*\mathbb{P}(C) \qquad \forall \, C\in\mathcal{X} $$

Expected Value with respect to $Z$

I am using this definition. $$ \mathbb{E}_{p(Z)}(Z) = \int_{\mathsf{Z}} Z(\omega_z) \,\,d Z_*\mathbb{P}(\omega_z) $$

Expected Value with respect to $X$

$$ \mathbb{E}_{p(X)}[X] = \int_{\mathsf{X}} X(\omega_x) \,\, d X_*Z_*\mathbb{P}(\omega_x) $$

Answer 1

Let $Z$ be a random variable with distribution $P^Z$, meaning that for any measurable set $A$,$$\mathbb P(Z\in A)=P^Z(A)$$ Then, for any measurable transform $f$, $X=f(Z)$ is a random variable with distribution $P^X$ such that, for any measurable set $A$,$$P^X(A)=\mathbb P(X\in A)=\mathbb P(f(Z)\in A)=\mathbb P(Z\in f^{-1}(A))=P^Z(f^{-1}(A))$$ where $$f^{-1}(A)=\{x;\ f(x)\in A\}$$ (which applies even when $f$ is not invertible).

This means that, when $g(\cdot)$ is an indicator function, $\mathbb I_A$, the equality \begin{align}\mathbb E^{P^X}[g(X)]&=\mathbb E^{P^X}[\mathbb I_A(X)]\\ &=\mathbb P^X(A)\\ &=\mathbb P^Z(f^{-1}(A)]\\ &=\mathbb E^{P^Z}[\mathbb I_{f^{-1}(A)}(Z)]\\ &=\mathbb E^{P^Z}[\mathbb I_A(f(Z))]=\mathbb E^{P^Z}[g(f(Z))] \end{align} stands. The conclusion follows (as usual) when writing any measurable function $g$ as a limit of weighted sums of indicator functions. The expectation under the push-forward measure $P^X$ is indeed the expectation of the $f$-transformed variate under the initial measure $P^Z$: $$\mathbb E^{P^X}[g(X)]=\mathbb E^{P^Z}[g(f(Z))]$$

Answer 2

Here's a simple case using random variable transformation. Assume $p_Z(z) = p_X(f(z)) \frac{df(z)}{dz}$ holds. Then we have, $$ \int g(f(z)) p_Z(z)dz = \int g(f(z)) p_X(f(z)) \frac{df(z)}{dz} dz = \int g(x) p_X(x) dx $$