Is the least square estimator unique?

Given $X\in\mathbb R^{n\times p}$ and $y\in \mathbb R^n$, the least square coefficients are: $\hat{\beta} = \text{argmin} \| X\beta - y\|^2_2$.

Is $\hat{\beta}$ unique in the case $\text{rank}(X)=p$?

• Yes, providing that $n \ge p$. The key is that $\beta$ is unique if the columns of $X$ are linearly independent, which $rank(X) = p$ combined with $n \ge p$ ensures. Apr 4 '16 at 0:22
• can you write a proof ? Apr 4 '16 at 0:23
• I could, but I leave that to you as an exercise. Apr 4 '16 at 0:24
• There's a algebraic-geometric proof if you know the theory of vector spaces. Are you familiar with vector spaces ? Apr 4 '16 at 10:40
• – whuber
Apr 4 '16 at 16:09

Assume $\text{rank}(X) = p$ and $n \geq p$ (in fact $\text{rank}(X) = p$ implies that $n \geq p$), the least square estimator has an explicit expression: $$\hat{\beta} = (X^TX)^{-1}X^Ty,$$ in which $X^TX$ is non-singular. Since the inverse is unique, $\hat{\beta}$ is unique.
So probably the only difficult part is to show that $X^TX$ is non-singular provided $\text{rank}(X) = p$. There are multiple ways to show that, perhaps the easiest way is to use $$\text{rank}(X^TX) = \text{rank}(X) \tag{*}$$ for any matrix $X$, also note that $X^TX$ is a $p \times p$ matrix, thus $X^TX$ has to be non-singular. Let me know if you need a proof for $(*)$.