Skip to main content
2 of 3
added 109 characters in body
Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k

I did this repeatedly, and after many observations, I ended up getting "perfect" fits in R. Why?

Because you have as many covariates as observations. In fact if you're not counting the constant column as one of the covariates it would even happen if you had one fewer covariates.

Is this an obvious result?

Yes.

Mark two points on a piece of paper. Draw the line of closest possible fit to those two points. Is it a perfect fit?

Now consider three points in space. Would the plane of best fit go through all three points? (Stick three fingers up in the air and try to lay a flat piece of cardboard on them -- can it touch all three fingers at the same time?)

If you have as many covariates (this time counting the constant predictor as one) as you do points, then the fit goes exactly through the data since each covariate has an additional degree of freedom with which to make an exact fit to an additional point*, there are no degrees of freedom left for error.

* this only fails to happen if your set of covariates (predictors) are linearly dependent, but that won't have been the case for your simulation.

Is the process useful in any way?

Well, hopefully it leads you to learning something pretty basic about regression (or indeed, about linear algebra), and knowledge you found yourself is certainly likely to be useful -- it's more likely to stick in the mind.

Glen_b
  • 290.5k
  • 37
  • 652
  • 1.1k