2
$\begingroup$

Possible Duplicate:
Calculating the confidence interval for simple linear regression coefficient estimates

I was referring to this wiki article http://en.wikipedia.org/wiki/Simple_linear_regression where it tried to calculate the confidence interval of the regression parameters

$$ \hat\alpha(intercept) $$ and

$$ \hat\beta(slope) $$

where

$$ (\alpha) (\beta) \\ $$ are the true population parameters. It then calculates the variance of beta as

$$ \sigma^2/\sum(x_i-x_m)^2\\ $$ where $$\sigma^2$$ is the variance of error term. I didn't get how this variance was calculated. Also I didn't get the rest of the derivation for the confidence interval. Any suggestions?

$\endgroup$
2
  • 1
    $\begingroup$ It would be preferable to incorporate this into your previous question. Have you searched the site for duplicate and related questions? $\endgroup$
    – cardinal
    Jun 30, 2012 at 22:13
  • $\begingroup$ Please update your previous question. I'm closing this one as a duplicate. $\endgroup$
    – chl
    Jul 1, 2012 at 10:07

2 Answers 2

0
$\begingroup$

The variance of the residuals is just the sum of the squared residuals (you don't need to subtract off the mean since the mean is already 0) divided by $n-p$ where p is the number of parameters estimated in the regression to get the residuals (2 if you estimated an intercept and one slope). In math notation:

$\hat\sigma^2 = \frac1{n-p}\sum r_i^2$

The general formula for the confidence interval (Wald interval) is the parameter estimate plus and minus a table value (that brings in the confidence level) times the standard error of that parameter. So you need a table value which will just be the t-table value with $n-p$ degrees of freedom and the square root of the above variance as the standard error.

$\endgroup$
8
  • $\begingroup$ By variance I mean variance of beta not the residual variance $\endgroup$
    – user34790
    Jun 30, 2012 at 23:10
  • $\begingroup$ The formula for the variance of beta is just what you have written above. Think back to the standard error of the mean for 1 sample cases. The variance version of that standard error is $\frac{\sigma^2}{n}$ The variance for beta is similar, but instead of dividing by $n$ the sample size the variation depends on the distribution of the predictor variable ($x$). If the x values are more spread out then there is less variability in the estimate, if the x values are all real close together then a small change in one y value will have a bigger effect on the estimate of the slope, hence the den. $\endgroup$
    – Greg Snow
    Jul 1, 2012 at 1:48
  • $\begingroup$ Yeah that's true but how do we know when to use which formula. I mean how this formula was derived? $\endgroup$
    – user34790
    Jul 1, 2012 at 13:57
  • $\begingroup$ You can derive the formula by starting with the equation for the slope $\frac{ \sum{(x_i - \bar{x}) y_i} }{ \sum{(x_i-\bar{x})^2} }$ and taking the variance of that. The y is the only variable, everything else is fixed, so if you use the rules about the variance of a sum and the variance of a constant times your variable then you find $\sigma^2$ as the variance of the $y_i$'s and all the constant pieces cancel until you are left with just the denominator above. $\endgroup$
    – Greg Snow
    Jul 2, 2012 at 21:26
  • $\begingroup$ I don't get it how come the equation of the slope is that $\endgroup$
    – user34790
    Jul 3, 2012 at 1:43
0
$\begingroup$

Because $\hat\beta = (X^TX)^{-1} X^T Y$ and $\mathrm{Var}(CY)=C^2 \mathrm{Var}(Y)$ the $X$s appear in the denominator of the variance estimate.

$\endgroup$
3
  • $\begingroup$ Your first expression has a term that's gone missing. This is also very good intuition but it's not really the explanation. $\endgroup$
    – cardinal
    Jun 30, 2012 at 22:49
  • $\begingroup$ Fixed. Because the β^1 component of the regression parameter as a linear combination of the ys the xs appear as they do in the formula. This is maybe handwavy rather than explicit but right nontheless. $\endgroup$ Jun 30, 2012 at 23:03
  • $\begingroup$ Yes. I guess what I was getting at was that the "reason" the $X$s appear in the denominator is through the determinant when taking the inverse of $X^T X$. But, it is good intuition that the inverse of the matrix acts much like a division would in the case of real numbers. (Of course, there are algebraic notions at play here, but they're beyond what is necessarily to address the problem at hand.) $\endgroup$
    – cardinal
    Jun 30, 2012 at 23:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.