Conditional Expectation and Prediction

Question

This is taken from the book "A First Course in Probability" from Sheldon Ross:

Sometimes a situation arises in which the value of a random variable $X$ is observed and then, on the basis of the observed value, an attempt is made to predict the value of a second random variable $Y$. Let $g(X)$ denote the predictor; that is, if $X$ is observed to equal $x$, then $g(x)$ is our prediction for the value of $Y$. Clearly, we would like to choose $g$ so that $g(X)$ tends to be close to $Y$. One possible criterion for closeness is to choose $g$ so as to minimize $E[(Y − g(X))^{2}]$. We now show that, under this criterion, the best possible predictor of $Y$ is $g(X) = E[Y|X]$.

So, I understand most of the proof that is provided in the book, but the start of the proof is what I do not understand:

So, why is ok to write the conditional expectation? We are trying to minimize the unconditional expectation, why can we assume that is the same as trying to minimize the conditional expectation on $X$?

Answer 1

I guess you want to understand why he began the proof with $\mathbb{E}[(Y-g(X))^2|X]$ instead of $\mathbb{E}[(Y-g(X))^2]$. By the Law of Iterated Expectations we have that

$$ \mathbb{E}[(Y-g(X))^2] = \mathbb{E}[\mathbb{E}[(Y-g(X))^2|X]]\quad.$$

So, if you want to minimize the first quantity, what you can do is, for each $X$, minimize $\mathbb{E}[(Y-g(X))^2|X]$. To be more rigorous, suppose that there is a function f such that

$$ \mathbb{E}[(Y-g(X))^2|X] \geq \mathbb{E}[(Y-f(X))^2|X]\quad.$$

Consequently, using the properties of expectation, it follows that

$$ \mathbb{E}[(Y-g(X))^2] = \mathbb{E}[\mathbb{E}[(Y-g(X))^2|X]] \geq \mathbb{E}[\mathbb{E}[(Y-f(X))^2|X]] = \mathbb{E}[(Y-f(X))^2] \quad.$$

This proves rigorously why you can focus on minimizing the conditional expectation.

Answer 2

In the end, the proof states the following: $$E[(Y-g(X))^2|X]=E[(Y-E[Y|X]^2)|X]+E[(E[Y|X]-g(X))^2|X]$$ If you take the expectation of both sides and then use the law of iterated expectations, i.e. $E[A]=E[E[A|B]]$, we have what we need: $$E[(Y-g(X))^2]=E[(Y-E[Y|X]^2)]+E[(E[Y|X]-g(X))^2]$$