7
$\begingroup$

I was wondering what to do with the following non-normal distribution of residuals of my multiple regression.

enter image description here

Normal Q_Q Plot of standardized residuals

Normality test of standardized residual

                    Shapiro-Wilk        
           Statistic: ,955 df: 131 Sig: ,000

According to the Shapiro-wilk test the normality test fails. However, after excluding the outlier located at the left bottom of the graph, I get these test results:

Normality test of standardized residual

                    Shapiro-Wilk        
           Statistic: ,980 df: 130 Sig: ,055

What are my options when the residuals don't have a normal distribution? Is it for example alright to remove outliers in order to achieve a normal distribution?

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ Do you have multiple X variables & a single Y variable, or do you have multiple Y variables? Note that the former is "multiple regression", & the latter is "multivariate regression". Under the assumption that your situation is of the former type, I changed your phrasing. $\endgroup$
    – gung - Reinstate Monica
    Jul 28, 2015 at 21:39
  • 2
    $\begingroup$ Residuals are never normally distributed, and I frankly don't think it matters much. See how well the model predicts new observations; that should be your criterion for how you judge the quality of fit. You might consider removing outliers to improve this fit, but not to achieve normality. $\endgroup$
    – dsaxton
    Jul 28, 2015 at 21:47
  • 3
    $\begingroup$ You can just go ahead and delete the outliers, so long as you have a crystal ball which magically recognizes when the real world situation would have an outlier, and therefore advises you not to apply your model to make predictions in such circumstances. Bonus points if you are using your fitted model for risk calculations, in which the behavior in the tails is all that matters - in such case, pesky outliers can be deleted, and now the risk of a bad outcome becomes very low and acceptable - that's what's called risk management. :) $\endgroup$
    – Mark L. Stone
    Jul 29, 2015 at 0:04
  • $\begingroup$ It is a multiple regression indeed, thanks for adjusting. @ Dsaxton, could you clarify "residuals are never normally distributed"? Because I believe this is a assumption that has to be met in a multiple regression? $\endgroup$
    – user83520
    Jul 29, 2015 at 10:29

2 Answers 2

Reset to default
17
$\begingroup$

  1. You should not remove outliers just because they make the distribution of the residuals non-normal. You may examine the case that has that high residual and see if there are problems with it (the easiest would be if it is a data entry error) but you must justify your deletion on substantive grounds.

  2. Assuming there is no good reason to remove that observation, you can run the regression with and without it and see if there are any large differences in the parameter estimates; if not, you can leave it in and note that removing it made little difference

  3. If it makes a big difference, then you could try robust regression, which deals with outliers or quantile regression, which makes no assumptions about the distribution of the residuals. I am a fan of quantile regression, which I think is very underutilized.

Improve this answer
$\endgroup$
4
$\begingroup$

@PeterFlom has made some good points here. I agree with his three points and his plan of action. Let me clear up one remaining issue:

You are correct to note that only the residuals need to be normally distributed. However, @dsaxton is also right that in the real world, no data (including residuals) are ever perfectly normal. Thus what you really need are residuals that are 'normal enough'. If the population distribution of errors is very close to normal (which is implied by your qq-plot once the outlier is accounted for), then the central limit theorem implies that the sampling distribution of your betas will converge to the normal as $N$ increases. So although your data are still nearly significant even with the outlier excluded, I think you will be fine following @PeterFlom's advice. You may be interested in reading this excellent CV thread: Is normality testing 'essentially useless'?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.