What to do if residuals are not normally distributed?

I was wondering what to do with the following non-normal distribution of residuals of my multiple regression.

Normality test of standardized residual

                    Shapiro-Wilk
Statistic: ,955 df: 131 Sig: ,000


According to the Shapiro-wilk test the normality test fails. However, after excluding the outlier located at the left bottom of the graph, I get these test results:

Normality test of standardized residual

                    Shapiro-Wilk
Statistic: ,980 df: 130 Sig: ,055


What are my options when the residuals don't have a normal distribution? Is it for example alright to remove outliers in order to achieve a normal distribution?

• Do you have multiple X variables & a single Y variable, or do you have multiple Y variables? Note that the former is "multiple regression", & the latter is "multivariate regression". Under the assumption that your situation is of the former type, I changed your phrasing. – gung - Reinstate Monica Jul 28 '15 at 21:39
• Residuals are never normally distributed, and I frankly don't think it matters much. See how well the model predicts new observations; that should be your criterion for how you judge the quality of fit. You might consider removing outliers to improve this fit, but not to achieve normality. – dsaxton Jul 28 '15 at 21:47
• You can just go ahead and delete the outliers, so long as you have a crystal ball which magically recognizes when the real world situation would have an outlier, and therefore advises you not to apply your model to make predictions in such circumstances. Bonus points if you are using your fitted model for risk calculations, in which the behavior in the tails is all that matters - in such case, pesky outliers can be deleted, and now the risk of a bad outcome becomes very low and acceptable - that's what's called risk management. :) – Mark L. Stone Jul 29 '15 at 0:04
• It is a multiple regression indeed, thanks for adjusting. @ Dsaxton, could you clarify "residuals are never normally distributed"? Because I believe this is a assumption that has to be met in a multiple regression? – user83520 Jul 29 '15 at 10:29

You are correct to note that only the residuals need to be normally distributed. However, @dsaxton is also right that in the real world, no data (including residuals) are ever perfectly normal. Thus what you really need are residuals that are 'normal enough'. If the population distribution of errors is very close to normal (which is implied by your qq-plot once the outlier is accounted for), then the central limit theorem implies that the sampling distribution of your betas will converge to the normal as $N$ increases. So although your data are still nearly significant even with the outlier excluded, I think you will be fine following @PeterFlom's advice. You may be interested in reading this excellent CV thread: Is normality testing 'essentially useless'?