# Correction for heavy-tailed distribution of residuals?

I'm interested in studying the effect of $$x$$ on $$y$$ using a fixed effects method. The residuals follow a heavy tail distribution, as the normal Q-Q plot suggests. For inference, I need a normal distribution of the residuals.

Which strategies can be applied to the model such that the residuals become more normally distributed? I thought about log-log transformations. This made the residuals indeed more normally distributed, but the heavy tails still persist.

• For inference, I need a normal distribution of the residuals. Why?
– Dave
Commented May 14 at 11:20
• @Dave To my understanding, this is a core OLS assumption which must hold for purposes of correct inference. Isn't this true?
– TFT
Commented May 14 at 11:26
• What is the nature of the outcome variable? For example, what do you think contibutes to it's skew in the real world? Perhaps that will illustrate why the tails act the way they do, and whether or not one should do something about it. Commented May 14 at 11:29
• Yes, if you demand exact inferences, but all models are wrong, so this is never going to be satisfied 100%. Fortunately, OLS tends to be rather robust to deviations from the Gaussian ideal, with the robustness being stronger for larger sample sizes. This robustness is not perfect and can be broken, but OLS is usually considered rather robust. As an alternative, you may consider proportional odds ordinal modeling, which does not make a Gaussian assumption and contains the Wilcoxon and Kruskal-Wallis tests as special cases.
– Dave
Commented May 14 at 11:29

Which strategies can be applied to the model such that the residuals become more normally distributed?

Make better predictions.

The heavy tails of your residuals exist because you do a poor job of predictig values. Residuals are large in magnitude because your model predictions badly miss the true values.

Knock it off! Make better predictions!

Now, it may be that, despite all of your efforts, you cannot tighten up the predictions past a certain point and that you are left with heavy tails. Fortunately, OLS is rather robust to deviations from the Gaussian ideal. This robustness is not perfect and can be broken (convergence theorems are, after all, limit theorems that do not apply to finite samples), but a large sample size is often enough to handle many deviations from the Gaussian ideal. A possible check of this is to bootstrap and check how your bootstrap parameter distributions compare to the theoretical distributions based on a Gaussian likelihood.

If that is not enough, there are a few alternatives.

1. Proportional odds ordinal models do not make a Gaussian assumption. These models generalize Wilcoxon and Kruskal-Wallis nonparametric tests.

2. Fit a model that minimizes absolute loss instead of square loss. Confidence intervals can be calculated using bootstrap. All of this is available in the R package quantreg. Frank Harrell has argued that this is not very efficient and tends to support proportional odds ordinal models, but this may be easier to interpret.

3. Use a method-of-moments or generalized method-of-moments estimator. Bootstrap is viable for calculating confidence intervals, and there might be alternatives if you scour the literature on (G)MM for fixed-effects regressions. (G)MM estimators are rather common in econometrics when the author does not want to commit to a likelihood for maximum likelihood estimation, such as you not wanting to commit to a Gaussian likelihood.

First, do think about the comments by Shawn and David (+1 to all those).

Second, if you decide that the residuals are so non-normal that you need to do something about it then, rather than transforming the variables, you can change the model. There are a few models that make no assumptions about the residuals at all. The "closest" to OLS regression are probably robust regression (there are a variety of types) and quantile regression.