Law of unconscious statistician for conditional expectation

Question

I have random variables $X$, $Y$ with joint distribution $f_{XY}(x,y)$ and conditional distribution $f_{X|Y}(x|y)$ and another random variable $Z=g(X)$ with $g$ being bijective is it true that

$$E(Z|Y=y)=\int_{-\infty}^{\infty}g(x)f_{X|Y}(x|y)dx$$

if so, does $g$ need to be bijective for this to hold in general? If not, is there a way to find $E(Z|Y=y)$ knowing just the joint and conditional probability functions for $X$ and $Y$?

Answer 1

Let's see this from a slightly different point of view. It may appear long-winded, but it's a mechanical application of the rules of the probability calculus. I'm not going to use random-variable notation (I prefer Jaynes's notation) but I hope the reasoning will be clear nevertheless.

By definition, $$ \mathrm{E}(z |\, y) := \int z \; \mathrm{p}(z |\, y) \;\mathrm{d}z\;. $$ Now let's see whether the conditional density $\mathrm{p}(z |\, y)\,\mathrm{d}z$ is determined by the information given in the problem.

We have $\mathrm{p}(x |\, y)\,\mathrm{d}x$. We also know that $z=g(x)$. This is equivalent to (a limit case of) probabilistic information. It means two things: first, $$ \mathrm{p}(z |\, x)\;\mathrm{d}z = \delta[z - g(x)]\;\mathrm{d}z $$ that is, if we know the value of $x$ then we also know the value of $z$ with perfect certainty. Note that this is true no matter what kind of function $g$ is, bijective or not. Second, $$ \mathrm{p}(z |\, x,y) \;\mathrm{d}z= \mathrm{p}(z |\, x) \;\mathrm{d}z\;, $$ because if $x$ is known, then knowledge of $y$ is irrelevant for ascertaining $z$ (otherwise $g$ would have been a function of $x$ and $y$, for example).

Now we can use the theorem of total probability: $$ \begin{align} \mathrm{p}(z |\, y) &= \int \mathrm{p}(z |\, x,y)\; \mathrm{p}(x |\, y)\;\mathrm{d}x \\ &= \int \mathrm{p}(z |\, x)\; \mathrm{p}(x |\, y)\;\mathrm{d}x \\ &= \int \delta[z - g(x)]\; \mathrm{p}(x |\, y)\;\mathrm{d}x \end{align} $$ where we have used the two previous equations.

Now we can replace the newly found expression for $\mathrm{p}(z |\, y)\;\mathrm{d}z$ in the definition of expectation: $$\begin{align} \mathrm{E}(z |\, y) &:= \int z \; \mathrm{p}(z |\, y) \;\mathrm{d}z \\ &= \int z \; \biggl\{\int \delta[z - g(x)]\; \mathrm{p}(x |\, y)\;\mathrm{d}x\biggr\} \;\mathrm{d}z \\ &= \int \biggl\{\int z \; \delta[z - g(x)]\;\mathrm{d}z\biggr\}\; \mathrm{p}(x |\, y)\;\mathrm{d}x \\ &=\int g(x)\; \mathrm{p}(x |\, y)\;\mathrm{d}x \end{align} $$ Which is the desired result. Of course the two integrals can only be swapped under some regularity assumptions about the density, which we have swept under the carpet (they're especially important if $\mathrm{p}(x |\, y)$ is a generalized function, for example).

Answer 2

As far as I understand, the transformation function should be bijective for the expectation result to hold true. If it is not the case then the distribution of x is not going to be uniquely defined. A simple example is $z = x^2$. For a given x, z is deterministic but we can't determine the probability mass function of x by knowing z. Please let me know if I am wrong and it can indeed be inferred in some way.