# Problem

Suppose I have a random variable $$z$$ following a distribution $$p(z)$$. Suppose I have a transformation

$$f(z) = x$$

that transforms the random variable $$z$$ into a new random variable $$x$$ with distribution $$p(x)$$. I have often seen the following result

$$\mathbb{E}_{p(x)}[g(x)] = \mathbb{E}_{p(z)}[g(f(z))].$$

In other words, the expectation with respect to a distribution $$p(x)$$ can be written in terms of the original distribution $$p(z)$$.

Is there proof of this? I think this should work even if $$f(z)$$ is not invertible and/or differentiable.

# My Set-Up for a Solution

I will describe my measure theory set up.

## Distribution of Z

Suppose we have two measurable spaces $$(\Omega, \mathcal{F})$$ and $$(\mathsf{Z}, \mathcal{Z})$$. The random variable $$Z$$ is a measurable mapping $$Z: \Omega \to \mathsf{Z}$$

such that the pre-image $$Z^{-1}(B)$$ of any $$\mathcal{Z}$$-measurable set $$B\in \mathcal{Z}$$ is also $$\mathcal{F}$$-measurable:

$$Z^{-1}(B) = \{\omega\in \Omega \, :\, Z(\omega) \in B \} \in \mathcal{F} \qquad \forall \, B \in \mathcal{Z}$$ Now the distribution of $$Z$$ is a push-forward measure. Suppose we have a probability space $$(\Omega, \mathcal{F}, \mathbb{P})$$. This means we can measure sets in $$\mathcal{F}$$. The push-forward or distribution for $$Z$$ is a way of measuring sets in $$\mathcal{Z}$$ via $$\mathbb{P}$$.

Basically, the distribution $$Z_*\mathbb{P}$$ assigns to sets $$B\in\mathbb{Z}$$ the same value as if we map $$B$$ back to $$\mathcal{F}$$ via $$Z^{-1}$$ first, and then we find its measure using $$\mathbb{P}$$.

$$(\mathbb{P} \circ Z^{-1})(B) = Z_*\mathbb{P}(B) \qquad \forall \, B\in \mathcal{Z}$$

## Distribution of X

Now, the new random variable $$X$$ is basically a function of the random variable $$Z$$ and therefore $$X$$ is also a random variable.

Consider the probability space $$(\mathsf{Z}, \mathcal{Z}, Z_*\mathbb{P})$$ for $$Z$$. Consider also a measurable function $$X: \mathsf{Z} \to \mathsf{X}$$ where $$(\mathsf{X}, \mathcal{X})$$ is a measurable space. This essentially defines the random variable $$X$$. Since we can measure sets in $$\mathcal{Z}$$ using $$Z_*\mathbb{P}$$ we would like to measure sets in $$\mathcal{X}$$ too. To do this, we define the distribution of $$X$$ to be a push-forward measure. Essentially, to give a measure to a set $$C\in \mathcal{X}$$ it maps it to $$\mathcal{Z}$$ via $$X^{-1}$$ and then measures it with the distribution $$Z_*\mathbb{P}$$.

$$(Z_*\mathbb{P} \circ X^{-1})(C) = X_*Z_*\mathbb{P}(C) \qquad \forall \, C\in\mathcal{X}$$

## Expected Value with respect to $$Z$$

I am using this definition. $$\mathbb{E}_{p(Z)}(Z) = \int_{\mathsf{Z}} Z(\omega_z) \,\,d Z_*\mathbb{P}(\omega_z)$$

## Expected Value with respect to $$X$$

$$\mathbb{E}_{p(X)}[X] = \int_{\mathsf{X}} X(\omega_x) \,\, d X_*Z_*\mathbb{P}(\omega_x)$$

• Could the law of unconscious statistician be not applied here? As in, if define a function $h(z)\equiv g(f(z))$ and then use the law? Commented Oct 21, 2020 at 11:35
• @Dayne Mmm Interesting.. but here there's an additional change: the expectation on the right-hand side is taken with respect to $p(z)$, while on the left-hand side is taken with respect to $p(x)$. Commented Oct 21, 2020 at 11:38
• @Euler_Salter: This is the so-called law of unconscious statistician, exactly. (The proof in Wikipedia is alas unnecessarily limited to the case of $g$ being invertible. The only constraint is actually that the transform $f$ is measurable in order for $X$ to be a random variable.) Commented Oct 21, 2020 at 12:00

Let $$Z$$ be a random variable with distribution $$P^Z$$, meaning that for any measurable set $$A$$,$$\mathbb P(Z\in A)=P^Z(A)$$ Then, for any measurable transform $$f$$, $$X=f(Z)$$ is a random variable with distribution $$P^X$$ such that, for any measurable set $$A$$,$$P^X(A)=\mathbb P(X\in A)=\mathbb P(f(Z)\in A)=\mathbb P(Z\in f^{-1}(A))=P^Z(f^{-1}(A))$$ where $$f^{-1}(A)=\{x;\ f(x)\in A\}$$ (which applies even when $$f$$ is not invertible).

This means that, when $$g(\cdot)$$ is an indicator function, $$\mathbb I_A$$, the equality \begin{align}\mathbb E^{P^X}[g(X)]&=\mathbb E^{P^X}[\mathbb I_A(X)]\\ &=\mathbb P^X(A)\\ &=\mathbb P^Z(f^{-1}(A)]\\ &=\mathbb E^{P^Z}[\mathbb I_{f^{-1}(A)}(Z)]\\ &=\mathbb E^{P^Z}[\mathbb I_A(f(Z))]=\mathbb E^{P^Z}[g(f(Z))] \end{align} stands. The conclusion follows (as usual) when writing any measurable function $$g$$ as a limit of weighted sums of indicator functions. The expectation under the push-forward measure $$P^X$$ is indeed the expectation of the $$f$$-transformed variate under the initial measure $$P^Z$$: $$\mathbb E^{P^X}[g(X)]=\mathbb E^{P^Z}[g(f(Z))]$$

• Thank you! I can see that you've used the definition of a push-forward measure. But how does this prove the equality between the expected values? The definition of expected value that I am using is this $$\mathbb{E}_{p(X)}[g(X)] = \int_{\mathsf{X}} g(X(\omega_x)) d P^X(\omega_x).$$ How does this relate to the other expected value? $$\mathbb{E}_{p(Z)}[g(f(Z))] = \int_{\mathsf{Z}} g(f(Z(\omega_z))) dP^Z(\omega_z)$$ Commented Oct 21, 2020 at 12:49
• @whuber I've added some more measure theory to ease the path towards a proof. Commented Oct 21, 2020 at 12:55
• That's fine, but it strikes me that you are asking a non-measure-theoretic question framed in terms of distributions rather than random variables: "the expectation with respect to a distribution $p(x)$ can be written in terms of the original distribution $p(z).$" Although introducing random variables is natural, the way you have asked the question strongly suggests you were looking for more direct proofs.
– whuber
Commented Oct 21, 2020 at 13:00
• (I cope with such situations by deleting my partial answer. Later I can edit it and undelete it.)
– whuber
Commented Oct 21, 2020 at 13:07

Here's a simple case using random variable transformation. Assume $$p_Z(z) = p_X(f(z)) \frac{df(z)}{dz}$$ holds. Then we have, $$\int g(f(z)) p_Z(z)dz = \int g(f(z)) p_X(f(z)) \frac{df(z)}{dz} dz = \int g(x) p_X(x) dx$$