# When people say “do analysis with and without outliers”, what do they mean?

When people say that outliers ($\neq$ experimental error) shouldn't be removed, but that analysis should be done both with and without them, do they mean that I should use the same model for both cases, and then report on potential differences? Even if the model with outliers is not satisfying the assumptions of e.g. normality?

Is that what people usually do? Or do people usually use different models?

Researchers may want to know if a result is being driven by a few extreme observations.

• If no, then what you do about outliers is arguably a moot point.
• If yes, then it matters that you take a reasonable/correct approach for your situation because it affects the result!

### Some extreme examples:

• If the outliers are clear measurement error (eg. negative weight), then it would be crazy/insane to publish a result that's driven by these bogus observations.
• If you're estimating average returns and the outliers are negative -100% returns (eg. firm bankruptcy), then excluding them would be dishonest/insane.

This question Is it OK to remove outliers from data? discusses the outlier detection in detail. As discussed there, I usually don't recommend removing outliers from data. However, in certain cases where you are sure that it is true outlier, e.g. negative age and weight, you could remove them.

Coming back to the question, you could be best served by asking the person who asked you analyze data with all the knowledge you get from the linked question above. If I have to answer that question, I'd say I will analyze both sets of data with same model and report the differences but would fight for including outliers in the analysis till my death.

• Ok, you've made it clear you'll be fighting to the death, but for what? "Outlier" means different things to different people. An extreme case...a case that exerts undue influence on estimates...a case that "breaks the rules" (is "impossible" under certain assumptions). I'm not sure "true outlier" fully clarifies things. – rolando2 Mar 24 '17 at 0:16
• Fighting to death for including outliers in analysis. As I have said, if samples are definite outliers like negative weight, those can be considered true outliers and removed. – discipulus Mar 24 '17 at 0:21
• Noting that removing those impossible outliers shouldn't be the end of the story. Impossible values imply a bug in the data pipeline, but that bug might also be causing possible-but-wrong values that are less easy to detect. Cleaning up the impossible values without further investigation risks concealing a data quality problem and leading readers to put too much faith in the results. – Geoffrey Brent Mar 24 '17 at 4:08