# Linear regression - variance error term

We have the model for the linear regression

$$y=X\beta + \varepsilon.$$

Why is $\text{Var}(\varepsilon)=\sigma^2$?

Given the fact that we don't know the distribution of the error term, $\varepsilon$, how can we make assumption on the variance?

In ordinary linear regression we do know the distribution of the error term $\varepsilon$, up to the single unknown parameter $\sigma^2$. Namely, our model is that the errors are drawn iid from the distribution $$\varepsilon_i \sim \mathcal N(0,\sigma^2).$$

We then estimate $\sigma^2$ along with the unknown $\beta$ coefficients.

• I see, thank you very much Jonathan. What's the reasoning behind setting $E(\varepsilon)=0$ ? Commented Jan 26, 2013 at 0:40
• The premise of the model is that $E(y) = X\beta$. But $E(y) = E(X\beta + \varepsilon) = X\beta+E(\varepsilon)$ (since $X\beta$ is not random), so we can only have $E(y) = X\beta$ if $E(\varepsilon)=0$. Commented Jan 26, 2013 at 0:47
• What do you mean by: "We then estaimte $\sigma^2$ along with the unkown coefficients"? In OLS, I only obtain values for the $\beta$ coefficients. I do not obtain a value $\sigma^2$. Commented Dec 29, 2016 at 12:27
• You should not always make the assumption the error term is normally distributed. You must exert caution when making assumptions about the error term. There is a substantial body of work in non-parametric modeling which offers convincing evidence to suggest it can be dangerous in estimation. Commented Apr 6, 2018 at 17:09

We should not make this assumption uncritically.

The error term $\varepsilon_i$ conditional on a particular $X$ value $X_i$, like any random variable, has a variance, usually written $\sigma_i^2$. There is no assumption here, it is just notation for that variance.

However, one of the assumptions of classical linear regression is that the error terms conditional on different $X$ values all have the same variance, that is, for any $X_i$ and $X_j$, $\sigma_i^2 = \sigma_j^2$. This assumption, known as homoscedasticity, may or may not be met for a particular model applied to a particular population. Before drawing conclusions from ordinary least squares (OLS) regression it is good practice to apply appropriate tests (or at least inspection of residuals) to assess whether this assumption is met. Where the assumption is met we are justified in using a common symbol, usually $\sigma^2$, for the common variance of the error terms.

Where the assumption is not met, that is, where there is heteroscedasticity, OLS regression is liable to give biased estimates of the variances of the regression coefficients. In that case weighted least squares is used to correct for the heteroscedasticity. Properly used, this has the effect of transforming the model in such a way that homoscedasticity is restored.