Uniqueness of $x'\beta$ even when $\mathbb{E}(x^Tx)$ is not invertible As discussed in user25658's answer to this question, when one wants to compute 
$$
\beta = \mathbb{E}(x^Tx)^{-1} \mathbb{E}(x^TY)
$$
but $\mathbb{E}(x^Tx)$ is not invertible, $\beta$ is not uniquely identified.  A characterization of all possible values of $\beta$ is given by 
$$\beta= \mathbb{E}(x^Tx)^g\mathbb{E}(x^Ty)$$
where $\mathbb{E}(x^Tx)^g$ is the generalized inverse of $\mathbb{E}(x^Tx)$. 
In Hansen, Econometrics, pg 34, section 2.18 (as of today's version on Hansen's webiste) it is written that $x^T\beta = x^T \mathbb{E}(x^Tx)^g\mathbb{E}(x^Ty) $ is however uniquely identified. 
This result is not completely counterintuitive to me. I understand that that this is matrix multiplication and that the fact that for every $x$ there may be multiple $\beta(x)$, say $\beta(x)_1 \neq \beta(x)_2$, does not imply $x^T\beta(x)_1 \neq x^T\beta(x)_2$.
Nevertheless, I have been unsuccessful in trying to prove the claim. Could someone help me with that?
 A: Note that when $ E [xx^T] = A$ is not invertible you actually want a solution to the equation
$$ A\beta=E [xx^T]\beta=E [xy]=c$$
Now if a solution exists it must have the form
$$ \hat {\beta}=A^{+}c + (I-A^{+}A)w $$
Where $ w $ is an arbitrary vector with the same dimension as $ c $, and $ A^{+} $ is the moore penrose pseudo inverse - can be calculated by using the spectral decomposition of $ A $.  Now what happens to a predicted value?  This is given as:
$$ x^T \hat {\beta}=x^TA^{+}c + (x^T-x^TA^{+}A)w $$
Now using the spectral decomposition we have
$$ A=FLF^T\implies A^{+}=FL^{+} F^T $$
This gives us
$$x^T\hat {\beta}=x^TFL^{+} F^Tc + x^TF(I-LL^{+} )F^Tw$$
Now this is basically doing a co-ordinate transform by the matrix $ F $.  $ x^{*}=F^Tx $ and $ c^{*}=F^Tc$ and $ w^{*}=F^Tw $.  Now we will basically have that 
$$x^T\hat {\beta}=\sum_{k=1}^{r}\frac {x^{*}_{k} c^{*}_{k}}{l_k} +\sum_{k=r+1}^{p}x^{*}_{k} w^{*}_{k}$$
Where $ r $ is the rank of $ A=E [xx^T] $ and $ p\geq r $ is the dimension of $ x $.  So the arbitrariness of the predictions is "siphoned" off into a $ p-r $ subspace.
To remove it completely we need to plug in the "sample" version of A, namely  $$ A_s=\frac {1}{n}\sum_{I=1}^{n} x_ix_i^T $$ and then we note that $ x_i^{*} $ is just the "principal component" score for the ith observation in the sample.  This means that $ x_{ik}^{*}=0 $ for $ k> r $.  So this means that the arbitrary choice for $ w $ only affects the betas, but not the fitted values.
Note that this is not true for "future" predictions, for $ x $ values that weren't observed in the sample.
Hope this helps.
