# What to do if residuals are not normally distributed?

I was wondering what to do with the following non-normal distribution of residuals of my multiple regression.

Normality test of standardized residual

                    Shapiro-Wilk
Statistic: ,955 df: 131 Sig: ,000


According to the Shapiro-wilk test the normality test fails. However, after excluding the outlier located at the left bottom of the graph, I get these test results:

Normality test of standardized residual

                    Shapiro-Wilk
Statistic: ,980 df: 130 Sig: ,055


What are my options when the residuals don't have a normal distribution? Is it for example alright to remove outliers in order to achieve a normal distribution?

• Do you have multiple X variables & a single Y variable, or do you have multiple Y variables? Note that the former is "multiple regression", & the latter is "multivariate regression". Under the assumption that your situation is of the former type, I changed your phrasing. Jul 28, 2015 at 21:39
• Residuals are never normally distributed, and I frankly don't think it matters much. See how well the model predicts new observations; that should be your criterion for how you judge the quality of fit. You might consider removing outliers to improve this fit, but not to achieve normality. Jul 28, 2015 at 21:47
• You can just go ahead and delete the outliers, so long as you have a crystal ball which magically recognizes when the real world situation would have an outlier, and therefore advises you not to apply your model to make predictions in such circumstances. Bonus points if you are using your fitted model for risk calculations, in which the behavior in the tails is all that matters - in such case, pesky outliers can be deleted, and now the risk of a bad outcome becomes very low and acceptable - that's what's called risk management. :) Jul 29, 2015 at 0:04
• It is a multiple regression indeed, thanks for adjusting. @ Dsaxton, could you clarify "residuals are never normally distributed"? Because I believe this is a assumption that has to be met in a multiple regression? Jul 29, 2015 at 10:29

1. You should not remove outliers just because they make the distribution of the residuals non-normal. You may examine the case that has that high residual and see if there are problems with it (the easiest would be if it is a data entry error) but you must justify your deletion on substantive grounds.

2. Assuming there is no good reason to remove that observation, you can run the regression with and without it and see if there are any large differences in the parameter estimates; if not, you can leave it in and note that removing it made little difference

3. If it makes a big difference, then you could try robust regression, which deals with outliers or quantile regression, which makes no assumptions about the distribution of the residuals. I am a fan of quantile regression, which I think is very underutilized.

@PeterFlom has made some good points here. I agree with his three points and his plan of action. Let me clear up one remaining issue:

You are correct to note that only the residuals need to be normally distributed. However, @dsaxton is also right that in the real world, no data (including residuals) are ever perfectly normal. Thus what you really need are residuals that are 'normal enough'. If the population distribution of errors is very close to normal (which is implied by your qq-plot once the outlier is accounted for), then the central limit theorem implies that the sampling distribution of your betas will converge to the normal as $N$ increases. So although your data are still nearly significant even with the outlier excluded, I think you will be fine following @PeterFlom's advice. You may be interested in reading this excellent CV thread: Is normality testing 'essentially useless'?

As this is an optional assumption, just ignore the normality of residuals and go ahead. In most sensitive cases, you can remove or replace the outliers with mean value or justified value to improve the condition.

• Welcome to CV, Tanveer. This advice goes against a great deal of literature and experience, so if you want it to stand (and avoid downvotes) it will be important to provide a firm rationale or a good reference to support it. The language is also imprecise and may be misleading: nobody makes assumptions about the residuals. The relevant assumption concerns the conditional distribution(s) of the responses, which are diagnosed by examining the residuals. And assumptions about those conditional distributions are crucial!
– whuber
Apr 1, 2023 at 15:37