I consider a multiple linear regression model with only two explanatory variables.

$\ y=\beta_0+\beta_1x_1+\beta_2x_2+\epsilon$

fulfilling the CLM properties.

From the moment restrictions




I want to show in a simple way that the model breaks down without using matrix algebra if I only have two observations ($n=2$).

I have managed to do it with a simple linear regression model (only one independent variable) with only one observation ($n=1$), where the moment restrictions are:



Since we have $n=1$ it follows that the moment restrictions can also be written as:



Now Isolating for $\hat{\beta_{0}}$ in the first equation:


And plugging this into the second equation I get:

$x_{1}\left(y_{1}-\left(y_{1}-\hat{\beta}_{1}x_{1}\right)-\hat{\beta}_{1}x_{1}\right)=0\Leftrightarrow x_1\cdot0=0$

Showing that things break down.

But I have really problems with doing it in a simple way in the multiple linear regression framework.

Hope that someone can help.

  • 3
    $\begingroup$ Is there any good reason why you are hampering yourself by not using matrix algebra? You're essentially asking to reproduce a basic theorem of linear algebra (about dimensions of subspaces) while demanding that we not use the tools or concepts of linear algebra. The exercise seems pointless. $\endgroup$ – whuber Feb 10 at 13:37
  • $\begingroup$ Not other than my linear algebra is not that good. But if you have a solution using linear algebra or alternatively can refer me to the basic theorem, it would be a great help. $\endgroup$ – Rasmus12 Feb 10 at 18:15
  • $\begingroup$ The basic theorem asserts the dimension of the image plus the dimension of the kernel of a linear map of vector spaces sum to the dimension of its domain. Translation: there is more than one solution to any set of $n$ simultaneous linear equations in $p$ variables when $n \lt p.$ $\endgroup$ – whuber Feb 10 at 22:24
  • $\begingroup$ Possibly an even more direct translation: Consider $n=2$. With $p=2$ (typically, a constant and a regressor) you can put a line through two points. Now suppose $p=3$. In that case, you fit a plane, and there are infinitely many different planes that share these two points. $\endgroup$ – Christoph Hanck Feb 11 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.