Linear regression as a projection I'm trying to refresh my memory as to the relationship between ordinary least squares and the linear algebra procedure of projection. (possibly highly related to Linear algebra attributes of a typical data set)
Starting with a model matrix, $\mathbf{X} = [\mathbf{1}, \vec{x_1}, \ldots, \vec{x_p}]$ (with the usual $n$ rows) and a response $Y$, we can define the hat matrix as a projection $\mathbf{H} = \mathbf{X}(\mathbf{X}^t \mathbf{X})^{-1}\mathbf{X}^t $. Then $\hat{Y} = \mathbf{H}Y$. In linear algebra, we define the column space, the row space, and the null space for a matrix.
Considering OLS as a projection:

*

*Is the row/column space defined for $\mathbf{X}$? Does the column space correspond to what you can call a "fitted space" or similar, i.e. on $\mathbb{R}^{p}$?

*Is the "null space" a basis defined by the vector of residuals, i.e. on $\mathbb{R}^{n-p}$?

*If the model matrix $\mathbf{X}$ contains a duplicate, do we consider the row space to actually be $\mathbb{R}^{n-1}$?

 A: Q: Is the row/column space defined for $\mathbf{X}$? Does the column space correspond to what you can call a "fitted space" or similar, i.e. on $\mathbb{R}^{p}$?A: Defined by $\mathbf{X}$ sounds better to me. I have never heard of a "fitted" space, but "fitting space" sounds like a reasonable term in a regression context. Also, since there are $p+1$ columns, it is a $p+1$-dimensional vector space unless there is a perfect multicollinearity. This vector space is not $\mathbb{R}^{p}$ or $\mathbb{R}^{p+1}$, but rather, it is a subset of $\mathbb{R}^{n}$.
Q: Is the "null space" a basis defined by the vector of residuals, i.e. on $\mathbb{R}^{n-p}$?A: The residual vector is a point in the (usually) $\{n-(p+1)\}$-dimensional vector space that is the orthogonal complement of the so-called "fitting space." It is not $\mathbb{R}^{n-p}$, but rather it is a subset of $\mathbb{R}^{n}$.
Q: If the model matrix $\mathbf{X}$ contains a duplicate, do we consider the row space to actually be $\mathbb{R}^{n-1}$?A: Unless there are other collinearities, the column vector space now has dimension $p$  (i.e., $(p+1) -1$). Again, it is not $\mathbb{R}^{n}$ or $\mathbb{R}^{n-1}$; rather, it is a subset of $\mathbb{R}^{n}$.
