6
$\begingroup$

I am trying to derive the expression for the variance of $\hat{\beta_0}$ in simple linear regression. I substitute $\bar{y} - \hat{\beta_1} \bar{x}$ for $\hat \beta_0$, but in the intermediate steps the covariance term $\text{Cov}(\bar{y}, \hat{\beta_1})$ comes up and I don't know how to deal with it. Any help would be appreciated!

$\endgroup$
3
$\begingroup$

I was trying to prove the same and struggled a bit but finally got to a solution. I know this was asked long back but there might be other people interested.

$$Cov (\bar{Y}, \hat{\beta_1}) = E[(\bar{Y} - E(\bar{Y}))(\hat{\beta}_1 - \beta_1)]\\ = E[(\beta_0 + \bar{X}\beta_1 + \bar{\epsilon} - \beta_0 - \bar{X} \beta_1) (\hat{\beta}_1 - \beta_1)]\\ = E(\bar{\epsilon} (\hat{\beta}_1 - \beta_1)]\\ = E(\bar{\epsilon} \hat{\beta}_1) - \beta_1 E(\bar{\epsilon})\\ = E(\bar{\epsilon} \hat{\beta}_1) $$ Now we know that $$ \hat{\beta}_1 = \sum_{i=1}^n \dfrac{(X_i - \bar{X})Y_i}{\sum(X_i - \bar{X})^2}$$ which we can write as $$ \hat{\beta}_1 = \sum_{i=1}^n c_i Y_i$$ for convenience. Now consider, $$ Cov(\bar{\epsilon}, \hat{\beta}_1) = E(\bar{\epsilon} \hat{\beta}_1)\\ = Cov(\sum c_i \epsilon_i , \bar{\epsilon})\\ = \sum_{i=1}^n \sum_{j=1}^n c_i\times \dfrac{1}{n} Cov(\epsilon_i, \epsilon_j) \\ \text{by independence we get,}\\ = \sum_{i=1}^n \dfrac{c_i}{n} \sigma^2\\ = \dfrac{\sigma^2}{n}\sum_i c_i = 0$$ This is because $$ c_i = \dfrac{(X_i - \bar{X})}{\sum_{j=1}^n (X_j - \bar{X})^2}\\ \sum_{i=1}^n c_i = \dfrac{1}{\sum_{j=1}^n (X_j - \bar{X})^2} \sum_{i=1}^n (X_i - \bar{X})\\ =\dfrac{1}{\sum_{j=1}^n (X_j - \bar{X})^2} (n\bar{X} - n\bar{X}) = 0 $$

$\endgroup$
  • $\begingroup$ ok I will do so.. $\endgroup$ – user111092 Aug 12 '16 at 1:55
0
$\begingroup$

$$ {\rm Cov}\bigg[\frac 1 n \sum y_i,\ \frac{\sum(x_i - \bar x)y_i}{\sum(x_i-\bar x)^2}\bigg] = \frac 1 n \times \frac{\sum(x_i-\bar x)\sigma^2}{\sum(x_i - \bar x)^2} = 1$$ it does not equal zero this way I believe

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.