How is the normality of error term related to the standard error that is computed by statistical software for a particular coefficient of a variable?

Question

1)My instructor says that because we assume the normality of the errors, we can calculate the correct standard error for the coefficient of a variable and further their t-statistics and p-values, but i fail to understand how normality of the error terms influences the standard errors. 2)Then the instructor says that, this isn't much of a problem even if they are not normally distributed because we can make them so by using CLT. But CLT says that if you have a population with mean μ and standard deviation σ and take sufficiently large random samples from the population with replacement , then the distribution of the sample means will be approximately normally distributed, it will not make the errors itself in the sample normally distributed, and what we want here is for the errors in the sample to be normally distributed right?

Please forgive me, if i go grossly wrong on some basic concept as i am a mere beginner and looking to make my foundation in regression strong, and hence asking such questions which may seem to others to be silly.

Answer 1

Your instructor's explanation is a bit misleading. The estimator of the coefficient vector is:

$$\hat{\beta} = (X'X)^{-1}X'Y.$$ Replacing $Y$ with $X\beta+\epsilon$, we get $$\hat{\beta} = (X'X)^{-1}X'(X\beta+\epsilon) = \beta+(X'X)^{-1}X'\epsilon.$$ Notice the first term, $\beta$, is simply a constant, so its variance is zero. Hence we have $$Var(\hat{\beta}) = Var((X'X)^{-1}X'\epsilon).$$ Here, we treat $X$ as constants, so this gives $$Var(\hat{\beta}) = (X'X)^{-1}X'Var(\epsilon)X(X'X)^{-1} = (X'X)^{-1}X'\sigma^2X(X'X)^{-1}= \sigma^2(X'X)^{-1}X'X(X'X)^{-1} = \sigma^2(X'X)^{-1}.$$ Hence, the variance of our estimator is $Var(\hat{\beta}) = \sigma^2(X'X)^{-1}.$ Notice we did not use any distributional assumptions about $\epsilon$ to arrive at this result.

The hypothesis testing, however, is based on the assumption of normal errors. This assumptions leads to the result that $$\frac{\hat{\beta}-{\beta}_0}{se(\hat{\beta})} \sim t(N-K),$$ where $N$ is the sample size, $K$ is the length of the vector $\beta$, and $\beta_0$ is the hypothesized value (by default, software tests the hypothesis that $\beta=0$, meaning $\beta_0$ is set to zero). However, the central limit theorem gives us asymptotic normality regardless of the distribution of the errors, so this assumption is not very important if the sample size is sufficiently large.