Regression function: derivation I am reading Section $2.4$ of The Elements of Statistical Learning (page $18$). The authors are looking for a function $f$ for predicting output variable $Y$ given values of the input $X$. Starting from the squared error loss, they derive the regression function
$$ f(x) = E(Y|X=x) $$
as a solution to an optimization problem. Unfortunately, I am not able to derive it and I do not understand the meaning (high-level understanding) of what they are doing (and why).
 A: The conditional expectation $\mathbb E(Y\mid X=x)$ calculates the expected value of $Y$, while accounting for additional information that we have - namely the input variable $X$.  
They probably motivated it the following way: Let $\hat \theta$ be any estimator for $Y$. Then the $MSE$ is given by
$$MSE=\mathbb E[(Y-\hat \theta)^2]=\mathbb E[Y^2]-2\hat\theta\mathbb E[Y]+\hat \theta^2.$$
We want to minimize the MSE, w.r.t. $\hat\theta$ - to find the minimum, we take the derivative and acquire $2\hat\theta-2\mathbb E[Y]$, which is zero for $\hat \theta=\mathbb E[Y]$.  
Hence, $\mathbb E[Y]$ is the estimator that minimizes the MSE, if we have no further information about $Y$. As we have an additional input variable $X$ that we can use to estimate $Y$, we could get better results by incorporating it in the expectations, right? We get
$$\mathbb E[(Y-\hat \theta)^2\mid X=x]=\mathbb E[Y^2\mid X=x]-2\hat\theta\mathbb E[Y\mid X=x]+\hat \theta^2,$$
which is minimized for $\mathbb E[Y\mid X=x] = f(x)$, your regression function. 
