1
$\begingroup$

Okay so I think I found a formula for the coefficient estimates but it is not very concise. It has like 6 sum of squares but it is in a single fraction so it is calculable. I was wondering what the simplest formula would be for estimating the coefficients for a linear regression. So say you had the regression equation: $$y = b_0 + b_1 (x-\overline{x}) + b_2 (z-\overline{z}) + \epsilon$$ where $\overline{x}$ and $\overline{z}$ are averages of $x$ and $z$. So (with no use of matrices) what is the simplest formula you can get that involves the first and second sample moments of $y$, $x$ and $z$ for the coefficient estimates ($\hat{b}_1$ and $\hat{b}_2$)?

I just minimized the residual squared and used $s_{xx}$, $s_{xz}$ etc. notation to find the formulas but as I said they are not very nice.

Improve this question
$\endgroup$
5
  • $\begingroup$ I know you say that you do not want to use matrices, but have you tried solving the matrix equations? If you do, you should arrive at the conclusion you are looking for. $\endgroup$
    – user25658
    Commented Sep 15, 2013 at 17:08
  • $\begingroup$ @BabakP it is not that I do not want to, but my university stopped using matrices for a while now so our courses do not teach it. $\endgroup$
    – Mauro Augusto
    Commented Sep 15, 2013 at 18:06
  • $\begingroup$ If that is the case, will you accept an answer with matrices? $\endgroup$
    – user25658
    Commented Sep 15, 2013 at 18:30
  • $\begingroup$ @BabakP I am sorry but that is completely useless to me, I do not understand it... (I have not learned how to use matrices to that extent in the field of econometrics). Thanks for trying though. $\endgroup$
    – Mauro Augusto
    Commented Sep 15, 2013 at 19:47
  • $\begingroup$ You can obtain formulas without using matrix arithmetic by using repeated "linear matching" as described at stats.stackexchange.com/a/46508. Better, though, would be to learn how to work with matrices: there are plenty of textbooks and online resources. Your education should not be limited to what your university offers! (I find it unbelievable, though, that your university does not teach this material, it is so basic. You might want to do more research into what is available where you are.) $\endgroup$
    – whuber ♦
    Commented Nov 15, 2013 at 19:41

2 Answers 2

Reset to default
1
$\begingroup$

Are these the "not very nice" formulas that you found?
(Note that $\hat{b}_1$ and $\hat{b}_2$ have the same denominator.)

$\hat{b}_1 = (s_{zz}\,s_{xy} - s_{xz}\,s_{zy})\,/\,(s_{xx}\,s_{zz} - s_{xz}^2)$

$\hat{b}_2 = (s_{xx}\,s_{zy} - s_{xz}\,s_{xy})\,/\,(s_{xx}\,s_{zz} - s_{xz}^2)$

$\hat{b}_0 = \overline{y}$

Improve this answer
$\endgroup$
0
$\begingroup$

Consider rewriting the equation into matrix form. To make notation simpler, I will define $x_i^*=x_i-\bar x$ and $z_i^*=z_i-\bar z$. Now we can rewrite $$y_i = b_0 + b_1 (x_i-\overline{x}) + b_2 (z_i-\overline{z}) + \epsilon_i$$ as the following:

$$\mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{\epsilon}$$ Now to obtain coefficient estimates for $b_1$ and $b_2$ we can solve the the least square estimator $$\hat\beta=(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}$$ If it help to see, we can explicitly write the matrix equation $$\mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{\epsilon}$$ as $$\begin{pmatrix} y_1\\ y_2\\ \vdots\\ y_n \end{pmatrix}=\begin{pmatrix} 1&x_1^*&z_1^*\\ 1&x_2^*&z_2^*\\ \vdots&\vdots&\vdots\\ 1&x_n^*&z_n^*\\ \end{pmatrix}\begin{pmatrix} b_0\\ b_1\\ b_2 \end{pmatrix}+\begin{pmatrix} \epsilon_1\\ \epsilon_2\\ \vdots\\ \epsilon_n\\ \end{pmatrix}$$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.