0
$\begingroup$

I am conducting separate regression models for two of my hypotheses. If I spot outliers in one case of conducting the regression model can I then bring them back for my second? Or do these outliers have to be removed from the dataset completely?

$\endgroup$
3
  • $\begingroup$ Plz help if you can! I am really stuck here :( as I don't want to influence the integrity of the results but I have to get this sorted soon! Thanks in advance $\endgroup$
    – Liz
    Mar 1, 2016 at 3:20
  • $\begingroup$ What is your definition of an outlier? $\endgroup$
    – John
    Mar 1, 2016 at 8:20
  • 2
    $\begingroup$ If you are unsure and in a hurry, report the analyses with and without the outliers. This helps readers to asses the extent to which the integrity of the results is affected. $\endgroup$
    – stijn
    Mar 1, 2016 at 9:14

2 Answers 2

2
$\begingroup$

There is no in principle answer to your question. However you need to give some thought to why you are removing the outliers. In general outliers are removed from a model because

a) They have a substantial impact on the results. (Or potentially do.) b) They are believed to be anomalous and potentially wrong (e.g. data capture errors) or generated by a different process.

Now in the case of (b) it may well be that they occur because of the model under consideration does not address these anomalous points whereas the second model does and then they could be included. However there are few situations in which seems likely to be true.

On balance if there is a good argument to remove them or the first hypothesis it seems unlikely that the same reasons won't be true in the second (unless of course the model involves different variables altogether.) The answer to your question should ultimately follow from the logic you used to determine the outliers in the first place and the arguments you mounted to justify their exclusion.

$\endgroup$
1
  • $\begingroup$ While (b) seems justifiable when there is very clear evidence (really this should be decided well before any analysis), many people would not really accept (a) as an acceptable reason. In many areas outlier removal is seen as an unacceptable way of tweaking the results until the desired result is achieved and (a) to me has that flavor. $\endgroup$
    – Björn
    Mar 1, 2016 at 6:25
1
$\begingroup$

This will depend on why you are removing the outliers. There are two basic reasons for removing them:

1) They are data entry errors or impossible values and the errors can't be corrected. (E.g. a 400 year old man)

2) They violate the assumptions of the model

In 1) they should be removed from the data set altogether. In 2) they probably shouldn't be removed at all - instead use a model that relaxes the assumptions such as robust regression or quantile regression. I wrote about these here. The paper uses SAS but can easily be adapted to other programs.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.