Expectation with respect to a transformed random variable Problem
Suppose I have a random variable $z$ following a distribution $p(z)$. Suppose I have a transformation
$$
f(z) = x
$$
that transforms the random variable $z$ into a new random variable $x$ with distribution $p(x)$. I have often seen the following result
$$
\mathbb{E}_{p(x)}[g(x)] = \mathbb{E}_{p(z)}[g(f(z))].
$$
In other words, the expectation with respect to a distribution $p(x)$ can be written in terms of the original distribution $p(z)$.
Is there proof of this? I think this should work even if $f(z)$ is not invertible and/or differentiable.
My Set-Up for a Solution
I will describe my measure theory set up.
Distribution of Z
Suppose we have two measurable spaces $(\Omega, \mathcal{F})$ and $(\mathsf{Z}, \mathcal{Z})$. The random variable $Z$ is a measurable mapping
$$
Z: \Omega \to \mathsf{Z}
$$
such that the pre-image $Z^{-1}(B)$ of any $\mathcal{Z}$-measurable set $B\in \mathcal{Z}$ is also $\mathcal{F}$-measurable:
$$
Z^{-1}(B) = \{\omega\in \Omega \, :\, Z(\omega) \in B \} \in \mathcal{F} \qquad \forall \, B \in \mathcal{Z}
$$
Now the distribution of $Z$ is a push-forward measure. Suppose we have a probability space $(\Omega, \mathcal{F}, \mathbb{P})$. This means we can measure sets in $\mathcal{F}$. The push-forward or distribution for $Z$ is a way of measuring sets in $\mathcal{Z}$ via $\mathbb{P}$.
Basically, the distribution $Z_*\mathbb{P}$ assigns to sets $B\in\mathbb{Z}$ the same value as if we map $B$ back to $\mathcal{F}$ via $Z^{-1}$ first, and then we find its measure using $\mathbb{P}$.
$$
(\mathbb{P} \circ Z^{-1})(B) = Z_*\mathbb{P}(B) \qquad \forall \, B\in \mathcal{Z}
$$
Distribution of X
Now, the new random variable $X$ is basically a function of the random variable $Z$ and therefore $X$ is also a random variable.
Consider the probability space $(\mathsf{Z}, \mathcal{Z}, Z_*\mathbb{P})$ for $Z$. Consider also a measurable function
$$
X: \mathsf{Z} \to \mathsf{X}
$$
where $(\mathsf{X}, \mathcal{X})$ is a measurable space. This essentially defines the random variable $X$. Since we can measure sets in $\mathcal{Z}$ using $Z_*\mathbb{P}$ we would like to measure sets in $\mathcal{X}$ too. To do this, we define the distribution of $X$ to be a push-forward measure. Essentially, to give a measure to a set $C\in \mathcal{X}$ it maps it to $\mathcal{Z}$ via $X^{-1}$ and then measures it with the distribution $Z_*\mathbb{P}$.
$$
(Z_*\mathbb{P} \circ X^{-1})(C) = X_*Z_*\mathbb{P}(C) \qquad \forall \, C\in\mathcal{X}
$$
Expected Value with respect to $Z$
I am using this definition.
$$
\mathbb{E}_{p(Z)}(Z) = \int_{\mathsf{Z}} Z(\omega_z) \,\,d Z_*\mathbb{P}(\omega_z)
$$
Expected Value with respect to $X$
$$
\mathbb{E}_{p(X)}[X] = \int_{\mathsf{X}} X(\omega_x) \,\, d X_*Z_*\mathbb{P}(\omega_x)
$$
 A: Let $Z$ be a random variable with distribution $P^Z$, meaning that for any measurable set $A$,$$\mathbb P(Z\in A)=P^Z(A)$$
Then, for any measurable transform $f$, $X=f(Z)$ is a random variable with distribution $P^X$ such that, for any measurable set $A$,$$P^X(A)=\mathbb P(X\in A)=\mathbb P(f(Z)\in A)=\mathbb P(Z\in f^{-1}(A))=P^Z(f^{-1}(A))$$
where
$$f^{-1}(A)=\{x;\ f(x)\in A\}$$
(which applies even when $f$ is not invertible).
This means that, when $g(\cdot)$ is an indicator function, $\mathbb I_A$, the equality
\begin{align}\mathbb E^{P^X}[g(X)]&=\mathbb E^{P^X}[\mathbb I_A(X)]\\
&=\mathbb P^X(A)\\
&=\mathbb P^Z(f^{-1}(A)]\\
&=\mathbb E^{P^Z}[\mathbb I_{f^{-1}(A)}(Z)]\\
&=\mathbb E^{P^Z}[\mathbb I_A(f(Z))]=\mathbb E^{P^Z}[g(f(Z))]
\end{align}
stands. The conclusion follows (as usual) when writing any measurable function $g$ as a limit of weighted sums of indicator functions. The expectation under the push-forward measure $P^X$ is indeed the expectation of the $f$-transformed variate under the initial measure $P^Z$:
$$\mathbb E^{P^X}[g(X)]=\mathbb E^{P^Z}[g(f(Z))]$$
A: Here's a simple case using random variable transformation. Assume $p_Z(z) = p_X(f(z)) \frac{df(z)}{dz}$ holds. Then we have,
$$
\int g(f(z)) p_Z(z)dz = \int g(f(z)) p_X(f(z)) \frac{df(z)}{dz} dz
= \int g(x) p_X(x) dx
$$
