Obtaining the possible least squares solutions when $X^TX$ is not invertible If $X^TX$ is not invertible, what is the set of solutions for the least squares estimator $\hat{\beta_1}$ in the below?
$Y_i = \beta_0 +\beta_1(x_i-\bar{x}) +\epsilon_i$
I got as far as writing out the normal equations, but not sure how i'd begin to solve the equations 
 A: You want all the possible solutions? Write the linear model in matrix form as 
$$ Y=X\beta + \epsilon, $$ and let the Moore-Penrose inverse of matrix $A$ be denoted by $A^+$. The normal equations for the least squares problem is 
$$ X^TX \beta= X^T Y$$, and if $X^TX$ is invertible then $\hat{\beta}=(X^TX)^{-1}X^TY$ is the unique solution. Otherwise, we can use the Moore-Penrose inverse to find the minimum norm solution $\beta^*=(X^TX)^+ X^TY$.
But in this case there are infinitely many other solutions. To find all we use the principle that all solutions of a linear system can be found as one particular solution of the inhomogeneous equation plus the general solution of the corresponding homogeneous equation. In this case that is $X^T X \beta=0$, and the solutions (on unknown $\beta$) constitutes the null space or kernel of $X^T X$, which is known to coincide with the kernel of $X$. 
So all solutions can be found by adding some vector in the null space of $X$ to $\beta^*$, which is our particular solution. The null space of $X$ can be described by using the singular value decomposition, SVD. If the rank of $X$ is $q$, this can be written as $X=U D V^T$ where $X$ is $n\times p$, $u$ is $n\times q$, $D$ is $q\times q$ diagonal with the positive singular values on the diagonal and $V$ is $p\times q$. $U, V$ are both column orthogonal. Then the null space of $X$ is $\{ \beta \colon U D V^T=0\}$, which can be reduced to $\{\beta \colon V^T\beta=0\}$. That is simply the orthogonal complement to the column space of $V$, which is a subspace in $\mathbb{R}^p$. 
A: EDITED based on remarks by @AdamO
If $X^TX$ is not invertible:


*

*there is no unique solution for $\beta_1$

*it means not all columns are linearly independent, for example two columns might be proportional to each other (i.e. one is superfluous, just the same measurement but with a different unit of measure). 

A: In an equation that you gave as example $Y_i=\beta_0+\beta_1(x_i-\bar x)+\varepsilon_i$, non invertibility of X'X means that $x_i$ is a constant, i.e. $x_i=\bar x$. The solution is then $\beta_0=\bar Y$ and $\beta_1$ any finite number (because $(x_i-\bar x)=0$). As you expect, it's a degenerated case with inifinite number of solutions.
