Correction for heavy-tailed distribution of residuals?

Question

I'm interested in studying the effect of $x$ on $y$ using a fixed effects method. The residuals follow a heavy tail distribution, as the normal Q-Q plot suggests. For inference, I need a normal distribution of the residuals.

Which strategies can be applied to the model such that the residuals become more normally distributed? I thought about log-log transformations. This made the residuals indeed more normally distributed, but the heavy tails still persist.

Answer 1

Which strategies can be applied to the model such that the residuals become more normally distributed?

Make better predictions.

The heavy tails of your residuals exist because you do a poor job of predictig values. Residuals are large in magnitude because your model predictions badly miss the true values.

Knock it off! Make better predictions!

Now, it may be that, despite all of your efforts, you cannot tighten up the predictions past a certain point and that you are left with heavy tails. Fortunately, OLS is rather robust to deviations from the Gaussian ideal. This robustness is not perfect and can be broken (convergence theorems are, after all, limit theorems that do not apply to finite samples), but a large sample size is often enough to handle many deviations from the Gaussian ideal. A possible check of this is to bootstrap and check how your bootstrap parameter distributions compare to the theoretical distributions based on a Gaussian likelihood.

If that is not enough, there are a few alternatives.

1. Proportional odds ordinal models do not make a Gaussian assumption. These models generalize Wilcoxon and Kruskal-Wallis nonparametric tests.

2. Fit a model that minimizes absolute loss instead of square loss. Confidence intervals can be calculated using bootstrap. All of this is available in the R package quantreg. Frank Harrell has argued that this is not very efficient and tends to support proportional odds ordinal models, but this may be easier to interpret.

3. Use a method-of-moments or generalized method-of-moments estimator. Bootstrap is viable for calculating confidence intervals, and there might be alternatives if you scour the literature on (G)MM for fixed-effects regressions. (G)MM estimators are rather common in econometrics when the author does not want to commit to a likelihood for maximum likelihood estimation, such as you not wanting to commit to a Gaussian likelihood.

Answer 2

First, do think about the comments by Shawn and David (+1 to all those).

Second, if you decide that the residuals are so non-normal that you need to do something about it then, rather than transforming the variables, you can change the model. There are a few models that make no assumptions about the residuals at all. The "closest" to OLS regression are probably robust regression (there are a variety of types) and quantile regression.