1
$\begingroup$

For multiple linear regression we have $n$ observations of the $k+1$ variables,

$$y_i = \beta_0 + \beta_1x_{i1}+ ... + \beta_kx_{ik} + \epsilon_i \text{ for i in 1,...n }$$

$n$ should be greater than $k$

Why is this?

$\endgroup$
2
  • $\begingroup$ Otherwise, the model might provide a perfect fit to the data. Two points determine a line; three determine a plane, etc. $\endgroup$
    – BruceET
    Mar 23, 2020 at 10:10
  • 1
    $\begingroup$ In addition to what BruceET said for $n = k $, if $n < k $, then you've got a undetermined problem with more variables than equations. $\endgroup$
    – mlofton
    Mar 23, 2020 at 13:09

1 Answer 1

1
$\begingroup$

It's still a linear system, but as noted in the comments, it's an underdetermined system and should be approached differently. When $n\geq k$, it can still be underdetermined if some equations are linearly dependent. But, if $n<k$, it surely is.

Usually in regression problems, there are enough number of points so that the system of equations have no solution, therefore we resort to approximate solutions that's able to generalise as well. However, in areas like computational genomics the situation can be the opposite (e.g. lots of genes, scarce patient data). Sparse regression is among the ways to tackle with this set of problems.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.