I am reading through the book An introduction to Statistical learning with Applications in R, and get stuck:

When doing linear regression, we can write the relation as

$$Y = \beta_0 + \beta_1X + \epsilon$$

where $$E(\epsilon)=0$$

And, we can define the residual sum of squares (RSS) as $$RSS = e_1^2 + e_2^2 + ... +e_n^2$$ where $e_i = y_i-\hat{y_i}$

To minimize the RSS, we can get

$$\hat{\beta_1} = \frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2}$$ $$\hat{\beta_0} = \bar{y}-\hat{\beta_1}\bar{x}$$

But, I cannot get the standard error with $\hat{\beta_0}$, which is (as the book says but I do not know how to prove it)

$$SE(\hat{\beta_0})^2 = var(\epsilon)[\frac{1}{n} + \frac{\bar{x}^2}{\sum_{i=1}^{n}(x_i - \bar{x})^2}]$$ $$SE(\hat{\beta_1})^2 = \frac{var(\epsilon)}{\sum_{i=1}^{n}(x_i - \bar{x})^2}$$

How to prove $SE(\hat{\beta_0})^2$ and $SE(\hat{\beta_1})^2$?

  • 1
    What about a self-study tag ;) ? – RUser4512 Sep 21 '15 at 11:42
up vote 0 down vote accepted

Hint:

Write $$\widehat{\beta_1}= \frac{\sum_{i=1}^n \left( x_i-\bar{x} \right)}{\sum_{i=1}^n \left(x_i-\bar{x} \right)^2} y_i $$

and you can check that these two expressions are equivalent, as the sum of mean deviations is zero. Since we are treating the predictors as fixed, you can use the properties of the variance to get what you want.

Now for the intercept, again using the standard rules of variance, we have

$$Var\left\{\widehat{\beta_0}\right\}=Var\left\{\bar{y} \right\}+\bar{x}^2 Var\left\{ \widehat{\beta}_1 \right\}-2\bar{x} Cov\left\{\bar{y},\widehat{\beta}_1 \right\}$$

But now note that

$$Cov\left\{\bar{y}. \widehat{\beta}_1\right\}=Cov\left\{\frac{1}{n}\sum_{i=1}^n y_i, \frac{\sum_{i=1}^n \left( x_i-\bar{x} \right)}{\sum_{i=1}^n \left(x_i-\bar{x} \right)^2} y_i \right\} $$

and since the $y_i$s are independent this reduces to zero. I leave to you the details of this computation. Thus the variance of $\widehat{\beta}_0$ is the sum of the first two components and if you plug in everything, you should get what you are looking for.

Hope this helps.

Keep in mind that both $\beta_0$ and $\beta_1$ are random variables and the standard errors are the square roots of the variances of these two random variables.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.