2
$\begingroup$

I understand how to find the coefficients of a bivariate regression and univariate regression w/o an intercept, i.e:

Univariate: Y = BX + e

Bols = 
 = (X'X)^(-1) X'Y 
 = (X'X)^(-1) X'(BX + e)
 = B + (X'X)^(-1) Xe                      If we take expectation/ e and X are independent 
 = B

Bivariate: Y = X1*C + X2*B + e

X1*M_X2*Y = X1*M_X2*X1*C + X1*M_X2*X2*B + X1*M_X2*e
X1*M_X2*Y = X1*M_X2*X1*C + X1*M_X2*e      Assuming X and e are independent
C =  (X1*M_X2*Y)* (X1*M_X2*X1*C)^(-1) 
B =  (X2*M_X1*Y)* (X2*M_X1*X2*C)^(-1)

But when there is an intercept I am pretty confused about what you are supposed to do when an intercept is included.

I started to do univariate and got:

Univariate: Y = a + Bx + e

Bols = 
 = (X'X)^(-1) X'Y 
 = (X'X)^(-1) X'(a + BX + e)
 = (X'X)^(-1) X'(a) + B + (X'X)^(-1)X'e  Take E, E[e|x] = 0                      
 = (X'X)^(-1) X'(a) + B 
   Bols is a vector: [a  B] [ E[X'X]^(-1) ]*E[X']   1]'

But I am not sure if this is correct?

Or what should be done if it is bivariate: Y= a + Cx1 + Bx2 + e ?

Thanks!

$\endgroup$
1
  • $\begingroup$ I answered this question in detail last night on a closely related thread at stats.stackexchange.com/questions/196807/…. Although they would be hard to search for, there are several dozen other posts that show how to construct the design matrix $X$--you could probably find some by exploring our regression-related questions. $\endgroup$
    – whuber
    Commented Feb 22, 2016 at 14:14

1 Answer 1

2
$\begingroup$

Actually, you already give all relevant formulae yourself. Suppose you have for $N$ individuals a regression $Y_i = a + cx_{1,i} + bx_{2,i} + e_i, \; 1\leq i \leq N$. Now rewrite $x_i = (1, x_{1,i}, x_{2,i})'$ and stack these $x_i's$ in $X$ as $X = (x_1', x_2', ... x_N')'$. That is, you build a regressor matrix $X$ where each row corresponds to an individual $i$ whose measurements of $x_1$, $x_2$ you have taken. The first column consists of $1$s and takes into account that you have an intercept. One can now rewrite for $Y = (Y_1, Y_2, ... Y_N)'$, $e = (e_1, ... e_N)'$ and $\beta = (a,c,b)$ your regression as $Y = X\beta + e$. (This single equation represents $N$ equations, one for each individual.) Then, as before, $(X'X)^{-1}X'Y = \hat{\beta} = (\hat{a}, \hat{c}, \hat{b})'$ is the OLS estimator.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.