Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

Possible Duplicate:
Calculating the confidence interval for simple linear regression coefficient estimates

I was referring to this wiki article http://en.wikipedia.org/wiki/Simple_linear_regression where it tried to calculate the confidence interval of the regression parameters

$$ \hat\alpha(intercept) $$ and

$$ \hat\beta(slope) $$

where

$$ (\alpha) (\beta) \\ $$ are the true population parameters. It then calculates the variance of beta as

$$ \sigma^2/\sum(x_i-x_m)^2\\ $$ where $$\sigma^2$$ is the variance of error term. I didn't get how this variance was calculated. Also I didn't get the rest of the derivation for the confidence interval. Any suggestions?

share|improve this question
1  
It would be preferable to incorporate this into your previous question. Have you searched the site for duplicate and related questions? – cardinal Jun 30 '12 at 22:13
Please update your previous question. I'm closing this one as a duplicate. – chl♦ Jul 1 '12 at 10:07

marked as duplicate by cardinal, Macro, chl♦ Jul 1 '12 at 10:07

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2 Answers

up vote 0 down vote

The variance of the residuals is just the sum of the squared residuals (you don't need to subtract off the mean since the mean is already 0) divided by $n-p$ where p is the number of parameters estimated in the regression to get the residuals (2 if you estimated an intercept and one slope). In math notation:

$\hat\sigma^2 = \frac1{n-p}\sum r_i^2$

The general formula for the confidence interval (Wald interval) is the parameter estimate plus and minus a table value (that brings in the confidence level) times the standard error of that parameter. So you need a table value which will just be the t-table value with $n-p$ degrees of freedom and the square root of the above variance as the standard error.

share|improve this answer
By variance I mean variance of beta not the residual variance – user34790 Jun 30 '12 at 23:10
The formula for the variance of beta is just what you have written above. Think back to the standard error of the mean for 1 sample cases. The variance version of that standard error is $\frac{\sigma^2}{n}$ The variance for beta is similar, but instead of dividing by $n$ the sample size the variation depends on the distribution of the predictor variable ($x$). If the x values are more spread out then there is less variability in the estimate, if the x values are all real close together then a small change in one y value will have a bigger effect on the estimate of the slope, hence the den. – Greg Snow Jul 1 '12 at 1:48
Yeah that's true but how do we know when to use which formula. I mean how this formula was derived? – user34790 Jul 1 '12 at 13:57
You can derive the formula by starting with the equation for the slope $\frac{ \sum{(x_i - \bar{x}) y_i} }{ \sum{(x_i-\bar{x})^2} }$ and taking the variance of that. The y is the only variable, everything else is fixed, so if you use the rules about the variance of a sum and the variance of a constant times your variable then you find $\sigma^2$ as the variance of the $y_i$'s and all the constant pieces cancel until you are left with just the denominator above. – Greg Snow Jul 2 '12 at 21:26
I don't get it how come the equation of the slope is that – user34790 Jul 3 '12 at 1:43
show 3 more comments
up vote 0 down vote

Because $\hat\beta = (X^TX)^{-1} X^T Y$ and $\mathrm{Var}(CY)=C^2 \mathrm{Var}(Y)$ the $X$s appear in the denominator of the variance estimate.

share|improve this answer
Your first expression has a term that's gone missing. This is also very good intuition but it's not really the explanation. – cardinal Jun 30 '12 at 22:49
Fixed. Because the β^1 component of the regression parameter as a linear combination of the ys the xs appear as they do in the formula. This is maybe handwavy rather than explicit but right nontheless. – Michael Chernick Jun 30 '12 at 23:03
Yes. I guess what I was getting at was that the "reason" the $X$s appear in the denominator is through the determinant when taking the inverse of $X^T X$. But, it is good intuition that the inverse of the matrix acts much like a division would in the case of real numbers. (Of course, there are algebraic notions at play here, but they're beyond what is necessarily to address the problem at hand.) – cardinal Jun 30 '12 at 23:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.