Questions about how to proceed when residuals of linear regression are not exactly normally distributed

Question

I have done a linear regression analysis on some data and I want to construct confidence intervals on the coefficients. I have read that it is necessary for the residuals to be normally distributed to be able to construct accurate confidence intervals for the coefficients. It seems like the residuals of my model are somewhat normally distributed, but would fit better with an Exponentially modified Gaussian distribution (there are a few outliers to the right shifting the mean of the normal distribution). I have a few questions on how to proceed:

1. Is it the residuals that need to be normally distributed or is it enough that the errors are assumed to be normally distributed because there is no logical explanation of how the measurement errors couldn't be in the data generation process?
2. Can I use the fact that I have fitted a better distribution in any way to create more accurate confidence intervals?
3. Does the central limit theorem come into play here? If my sample size is large enough, are the confidence intervals accurate regardless if the residuals are not exactly normally distributed?
4. Can I create another model, a generalized linear model, with gaussian family and some link function to make the residuals normally distributed giving better accuracy for the confidence intervals? Or am I misunderstand the use of GLMs.

Answer 1

Regarding 1., no. You checked your assumptions. They were wrong.

Regarding 2., there are a number of things that could cause non-normal residuals. There could be outliers or influential points; the model could be mis-specified (e.g. an omitted variable, or a wrong form of relationship); the DV could be skewed; etc.)

Regarding 3., I steer clear of discussions about the CLT. Someone who has more math than I do can answer.

Regarding 4., what you can or should do depends on what caused the problem (see #2).

Answer 2

Your model is $Y_i=\boldsymbol{x}_i'\boldsymbol\beta + \epsilon_i$, for $i=1,\ldots,n$. The minimal set of assumptions we make about the errors is that they have zero mean, constant variance (i.e. the same for all $i$), and are uncorrelated. If $\boldsymbol\epsilon=(\epsilon_1,\ldots,\epsilon_n)'$, we can write these as $\mathbb{E}(\boldsymbol\epsilon)=0$ and $\mathrm{Var}(\boldsymbol\epsilon)=\sigma^2 \boldsymbol I_n$.

1. In addition to above assumptions, you may also wish to assume $\boldsymbol\epsilon \sim N(0, \sigma^2 \boldsymbol I_n)$, i.e. that the errors are normally distributed. If the model is correctly specified and the errors are normally distributed, then we can show that the residuals are also normally distributed. In your case the residuals are non-normal, so at least one of these assumptions does not hold (correct model specification and/or normal errors). Is this a problem? Not necessarily, though it means that we should be suspicious of any results that are also based on the assumption of normality (confidence intervals for the parameter estimates, $t$-tests, $p$-values, etc.).

2. I would start by checking any issues to do with the model structure. The answer by @peter-flom contains a thorough list of suggestions.

3. Yes. If the model is correctly specified and the errors satisfy the minimal set of assumptions (i.e. excluding normality), then the least-squares estimator of $\boldsymbol\beta$ is approximately normally distributed, so we can construct CIs in the usual way.

4. Again, I agree with @peter-flom. Fix what you can first. Also, you mention that you have some large positive residuals; is there no explanatory variable that can help identify these observations? An omitted variable could easily explain this behaviour.