Assume the true underlying linear approximation of a set of data is equal to $Y=2+3X +\epsilon$ where $\epsilon$ represents the irreducible error that is inherent in a linear approximation. I then perform a linear regression and arrive at my $\hat{\beta}_0$ and $\hat{\beta}_1$ parameters. In order to determine the standard error of $\hat{\beta}_0$ and $\hat{\beta}_1$, I use the following formulas:

$$ SE(\hat{\beta}_0)^2= \sigma^2 \big{[} \frac{1}{n} + \frac{\bar{x}^2}{\sum_{i=1}^n(x_i-\bar{x})^2} \big{]} $$

$$ SE(\hat{\beta}_1)^2= \frac{\sigma^2}{\sum_{i=1}^n(x_i-\bar{x})^2} $$

where $\sigma^2 = Var(\epsilon)$

Why does $\sigma^2 = Var(\epsilon)$ and not $\sigma^2 = Var(x)$ (the variance of my sample population)?

Where does $\epsilon$ work its way in to the derivation of the standard errors of these two parameters? $\epsilon$ has nothing to do with the least squares approach for calculating $\hat{\beta}_0$ and $\hat{\beta}_1$ in the first place. Also, how do we determine $\epsilon$? Doesn't $\epsilon$ require knowing that the "true" linear approximation of the data is $Y=2+3X +\epsilon$? We obviously would not know this in real life.

EDIT: one last question: what does it mean to assume that the errors $\epsilon_i$ for each observation are uncorrelated with common variance $\sigma^2$?

  • $\begingroup$ Ponder the different meanings of "$Var$" you are using: in $\sigma^2=\operatorname{Var}(\epsilon)$ the variance is a property of a random variable whereas in $\sigma^2=\operatorname{Var}(x)$ the variance measures the spread of a set of regressor values. $\endgroup$ – whuber Oct 5 '18 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.