1)My instructor says that because we assume the normality of the errors, we can calculate the correct standard error for the coefficient of a variable and further their t-statistics and p-values, but i fail to understand how normality of the error terms influences the standard errors. 2)Then the instructor says that, this isn't much of a problem even if they are not normally distributed because we can make them so by using CLT. But CLT says that if you have a population with mean μ and standard deviation σ and take sufficiently large random samples from the population with replacement , then the distribution of the sample means will be approximately normally distributed, it will not make the errors itself in the sample normally distributed, and what we want here is for the errors in the sample to be normally distributed right?

Please forgive me, if i go grossly wrong on some basic concept as i am a mere beginner and looking to make my foundation in regression strong, and hence asking such questions which may seem to others to be silly.


1 Answer 1


Your instructor's explanation is a bit misleading. The estimator of the coefficient vector is:

$$\hat{\beta} = (X'X)^{-1}X'Y.$$ Replacing $Y$ with $X\beta+\epsilon$, we get $$\hat{\beta} = (X'X)^{-1}X'(X\beta+\epsilon) = \beta+(X'X)^{-1}X'\epsilon.$$ Notice the first term, $\beta$, is simply a constant, so its variance is zero. Hence we have $$Var(\hat{\beta}) = Var((X'X)^{-1}X'\epsilon).$$ Here, we treat $X$ as constants, so this gives $$Var(\hat{\beta}) = (X'X)^{-1}X'Var(\epsilon)X(X'X)^{-1} = (X'X)^{-1}X'\sigma^2X(X'X)^{-1}= \sigma^2(X'X)^{-1}X'X(X'X)^{-1} = \sigma^2(X'X)^{-1}.$$ Hence, the variance of our estimator is $Var(\hat{\beta}) = \sigma^2(X'X)^{-1}.$ Notice we did not use any distributional assumptions about $\epsilon$ to arrive at this result.

The hypothesis testing, however, is based on the assumption of normal errors. This assumptions leads to the result that $$\frac{\hat{\beta}-{\beta}_0}{se(\hat{\beta})} \sim t(N-K),$$ where $N$ is the sample size, $K$ is the length of the vector $\beta$, and $\beta_0$ is the hypothesized value (by default, software tests the hypothesis that $\beta=0$, meaning $\beta_0$ is set to zero). However, the central limit theorem gives us asymptotic normality regardless of the distribution of the errors, so this assumption is not very important if the sample size is sufficiently large.

  • $\begingroup$ dlnB, I'm sorry but I couldn't understand your answer, with the vectors and others things, is there any way you could phrase it in a layman's language? Again, very sorry $\endgroup$ Commented Apr 29, 2020 at 9:50
  • $\begingroup$ The basic idea is that the hypothesis tests are based on a distribution (t-distribution). The t-distributed test statistics follow from the assumption of normally distributed errors. Even if this assumption is violated, asymptotic normality makes these tests valid with a sufficiently large sample. Calculation of the standard errors requires no distributional assumptions. $\endgroup$
    – dlnB
    Commented May 1, 2020 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.