Can an LS estimator be unbiased if there are infinite solutions? Let 
$$\mathbf{y} =\mathbf{A}\mathbf{x} + \mathbf{\eta} $$
where $\mathbf{\eta}$ is a vector of samples of white, zero-mean Gaussian noise. I want to estimate $\mathbf{x}$ (which is deterministic) given $\mathbf{y}$ using the least-squares method.
If the problem has a unique solution, then the estimator $\mathbf{\hat{x}} =(\mathbf{A}^T\mathbf{A})^{-1}\mathbf{A}^T\mathbf{y} $ is unbiased.
However, if $\mathbf{A}$ is doesn't have full rank, then there are infintely many solutions $\mathbf{\hat{x}}$. Are these (or some of them, or one of them) unbiased as well? Is there a proof regarding this topic that I could read?
 A: If $\mathbf A$ doesn't have full rank, then the model is not identifiable, which implies that an unbiased estimator does not exist. 
Here is what the proof looks like in this specific scenario. If $\mathbf A$ doesn't have full rank, then there exist two distinct vectors $\mathbf x^{(1)}$ and $\mathbf x^{(2)}$ such that $\mathbf A\mathbf x^{(1)} = \mathbf A\mathbf x^{(2)}$. In this case, the data $\mathbf y = \mathbf A\mathbf x + \eta$ has the same distribution under both $\mathbf x = \mathbf x^{(1)}$ and $\mathbf x = \mathbf x^{(2)}$. Suppose then that an unbiased estimator $T(\mathbf y)$ were to exist. Then by the definition of $T(\mathbf y)$ being unbiased, we must have $E_{\mathbf x^{(1)}}(T(\mathbf y)) = \mathbf x^{(1)}$, where $E_{\mathbf x^{(1)}}$ denotes the expected value taken under the scenario where $\mathbf x=\mathbf x^{(1)}$. Likewise $E_{\mathbf x^{(2)}}(T(\mathbf y)) = \mathbf x^{(2)}$. And yet, since $\mathbf y$ has the same distribution whether $\mathbf x = \mathbf x^{(1)}$ or $\mathbf x = \mathbf x^{(2)}$, it follows that $T(\mathbf y)$ also has the same distribution in both of these scenarios, which means that $E_{\mathbf x^{(1)}}(T(\mathbf y)) = E_{\mathbf x^{(2)}}(T(\mathbf y))$. Putting this together gives $\mathbf x^{(1)} = \mathbf x^{(2)}$, which is a contradiction since $\mathbf x^{(1)}$ and $\mathbf x^{(2)}$ were chosen to be distinct.
