How to get the Standard Error of linear-regression parameters?

Question

I am reading through the book An introduction to Statistical learning with Applications in R, and get stuck:

When doing linear regression, we can write the relation as

$$Y = \beta_0 + \beta_1X + \epsilon$$

where $$E(\epsilon)=0$$

And, we can define the residual sum of squares (RSS) as $$RSS = e_1^2 + e_2^2 + ... +e_n^2$$ where $e_i = y_i-\hat{y_i}$

To minimize the RSS, we can get

$$\hat{\beta_1} = \frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^2}$$ $$\hat{\beta_0} = \bar{y}-\hat{\beta_1}\bar{x}$$

But, I cannot get the standard error with $\hat{\beta_0}$, which is (as the book says but I do not know how to prove it)

$$SE(\hat{\beta_0})^2 = var(\epsilon)[\frac{1}{n} + \frac{\bar{x}^2}{\sum_{i=1}^{n}(x_i - \bar{x})^2}]$$ $$SE(\hat{\beta_1})^2 = \frac{var(\epsilon)}{\sum_{i=1}^{n}(x_i - \bar{x})^2}$$

How to prove $SE(\hat{\beta_0})^2$ and $SE(\hat{\beta_1})^2$?

Answer 1

Hint:

Write $$\widehat{\beta_1}= \frac{\sum_{i=1}^n \left( x_i-\bar{x} \right)}{\sum_{i=1}^n \left(x_i-\bar{x} \right)^2} y_i $$

and you can check that these two expressions are equivalent, as the sum of mean deviations is zero. Since we are treating the predictors as fixed, you can use the properties of the variance to get what you want.

Now for the intercept, again using the standard rules of variance, we have

$$Var\left\{\widehat{\beta_0}\right\}=Var\left\{\bar{y} \right\}+\bar{x}^2 Var\left\{ \widehat{\beta}_1 \right\}-2\bar{x} Cov\left\{\bar{y},\widehat{\beta}_1 \right\}$$

But now note that

$$Cov\left\{\bar{y}. \widehat{\beta}_1\right\}=Cov\left\{\frac{1}{n}\sum_{i=1}^n y_i, \frac{\sum_{i=1}^n \left( x_i-\bar{x} \right)}{\sum_{i=1}^n \left(x_i-\bar{x} \right)^2} y_i \right\} $$

and since the $y_i$s are independent this reduces to zero. I leave to you the details of this computation. Thus the variance of $\widehat{\beta}_0$ is the sum of the first two components and if you plug in everything, you should get what you are looking for.

Hope this helps.

Answer 2

Keep in mind that both $\beta_0$ and $\beta_1$ are random variables and the standard errors are the square roots of the variances of these two random variables.