6
$\begingroup$

Why is it necessary to have the number of independent variables less than the number of data points in a regression setup? If suppose, it is the other way round, i.e. the number of independent variables far exceed the number of observations, then what should be the way out?

$\endgroup$

1 Answer 1

13
$\begingroup$

Consider one of the simplest possible case. One independent variable (so 2 parameters, including the constant). One data point.

Plot your one data point

Draw a straight line through that one point. Draw a different straight line through the same point. Draw a third one. ... and so on.

$\hspace{3cm}$enter image description here

They all fit the data perfectly. Which one are you going to pick?

The problem is similar with two points and two predictors (with a plane through two points it's a bit like trying to rest a sheet of plywood on top of a picket fence - stable in one direction, but it's a see-saw in the other).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.