# Confusion in linear regression confidence interval calculation [duplicate]

I was referring to this wiki article http://en.wikipedia.org/wiki/Simple_linear_regression where it tried to calculate the confidence interval of the regression parameters

$$\hat\alpha(intercept)$$ and

$$\hat\beta(slope)$$

where

$$(\alpha) (\beta) \\$$ are the true population parameters. It then calculates the variance of beta as

$$\sigma^2/\sum(x_i-x_m)^2\\$$ where $$\sigma^2$$ is the variance of error term. I didn't get how this variance was calculated. Also I didn't get the rest of the derivation for the confidence interval. Any suggestions?

-
It would be preferable to incorporate this into your previous question. Have you searched the site for duplicate and related questions? –  cardinal Jun 30 '12 at 22:13
Please update your previous question. I'm closing this one as a duplicate. –  chl Jul 1 '12 at 10:07

## marked as duplicate by cardinal♦, Macro, chl♦Jul 1 '12 at 10:07

The variance of the residuals is just the sum of the squared residuals (you don't need to subtract off the mean since the mean is already 0) divided by $n-p$ where p is the number of parameters estimated in the regression to get the residuals (2 if you estimated an intercept and one slope). In math notation:

$\hat\sigma^2 = \frac1{n-p}\sum r_i^2$

The general formula for the confidence interval (Wald interval) is the parameter estimate plus and minus a table value (that brings in the confidence level) times the standard error of that parameter. So you need a table value which will just be the t-table value with $n-p$ degrees of freedom and the square root of the above variance as the standard error.

-
By variance I mean variance of beta not the residual variance –  user34790 Jun 30 '12 at 23:10
The formula for the variance of beta is just what you have written above. Think back to the standard error of the mean for 1 sample cases. The variance version of that standard error is $\frac{\sigma^2}{n}$ The variance for beta is similar, but instead of dividing by $n$ the sample size the variation depends on the distribution of the predictor variable ($x$). If the x values are more spread out then there is less variability in the estimate, if the x values are all real close together then a small change in one y value will have a bigger effect on the estimate of the slope, hence the den. –  Greg Snow Jul 1 '12 at 1:48
Yeah that's true but how do we know when to use which formula. I mean how this formula was derived? –  user34790 Jul 1 '12 at 13:57
You can derive the formula by starting with the equation for the slope $\frac{ \sum{(x_i - \bar{x}) y_i} }{ \sum{(x_i-\bar{x})^2} }$ and taking the variance of that. The y is the only variable, everything else is fixed, so if you use the rules about the variance of a sum and the variance of a constant times your variable then you find $\sigma^2$ as the variance of the $y_i$'s and all the constant pieces cancel until you are left with just the denominator above. –  Greg Snow Jul 2 '12 at 21:26
I don't get it how come the equation of the slope is that –  user34790 Jul 3 '12 at 1:43
Because $\hat\beta = (X^TX)^{-1} X^T Y$ and $\mathrm{Var}(CY)=C^2 \mathrm{Var}(Y)$ the $X$s appear in the denominator of the variance estimate.
Yes. I guess what I was getting at was that the "reason" the $X$s appear in the denominator is through the determinant when taking the inverse of $X^T X$. But, it is good intuition that the inverse of the matrix acts much like a division would in the case of real numbers. (Of course, there are algebraic notions at play here, but they're beyond what is necessarily to address the problem at hand.) –  cardinal Jun 30 '12 at 23:30