# Why are we not able to find standard error estimates in a linear regression model where there are more predictors than observations?

Suppose we have that $n << p$ and that we have a linear regression model where the predictors outnumber the observations. In this case, usually in statistical software packages, the standard errors and hence the confidence intervals show as NaN. The formal standard error formula is given by:

$$s_{\hat\beta_j}=\sqrt{\frac{\sum_i\hat\epsilon_i^2}{(n-p-1)\sum_i(x_i-\bar x)^2}}$$

It seems here that the only thing preventing the estimate is the $(n-p-1)$ on the bottom causing an invalid negative value. I understand it is there due to unbiasedness reasons. However, why can't be just remove $(n-p-1)$ from the term otherwise?

• You cannot generally create a valid formula just by removing the part that offends you. The ludicrousness of dividing by $n-p-1$ when $p \ge n-1$ is a strong signal that something is fundamentally wrong in that case. Indeed, assuming the predictors (plus the constant vector) are of full rank, you can perfectly fit the response, leaving no way to estimate the error. Reporting NaN sounds like the perfect way to express that ineluctable fact. – whuber Oct 17 '17 at 13:23
• As whuber says you get a perfect fit which means the standard error is 0. – Michael Chernick Oct 17 '17 at 14:20