I looked for a way to remove outliers from a dataset and I found this question.

In some of the comments and answers to this question, however, people mentioned that it is bad practice to remove outliers from the data.

In my dataset I have several outliers that very likely are just due to measurement errors. Even if some of them are not, I have no way of checking it case by case, because there are too many data points. Is it statistically valid than just to remove the outliers? Or, if not, what could be another solution?

If I just leave those points there, they influence e.g. the mean in a way that does not reflect reality (because most of them are errors anyway).

EDIT: I am working with skin conductance data. Most of the extreme values are due to artifacts like somebody pulling on the wires.

EDIT2: My main interest in analyzing the data is to determine if there is a difference between two groups

  • 3
    $\begingroup$ And what do you want to do? Data summary? Predictive analysis? Data visualization? Proving that there is (no) significant difference between two groups? As with all data cleaning, there is no general answer. $\endgroup$ Commented Mar 8, 2016 at 14:04
  • 5
  • 5
    $\begingroup$ I'm an engineer who works with lots of statistics. That was a disclaimer and a confession that means I have to deliver products. We are only allowed to remove fully attributed "bad" points. Can you prove it was from someone pulling a wire? If you get several intentional measures, you can bound and cluster there. Then you can split the data on the cluster (pull vs non-pull) and it isn't about outliers anymore. If you can't prove what the root cause is, you must (must) retain it. It speaks to variation, and that is a big chunk of analysis. You can't get rid of it if you don't like it. $\endgroup$ Commented Mar 8, 2016 at 17:26
  • 4
    $\begingroup$ I think you start by the wrong end. The first question is how do you identify the outliers in the first place? $\endgroup$
    – user603
    Commented Mar 8, 2016 at 20:44
  • 6
    $\begingroup$ Rather than arbitrary removal of arbitrarily identified outliers, you may be better to consider something like "since I have contamination from issues such as people pulling on wires, what methodologies can I use which are not badly affected by such contamination?" $\endgroup$
    – Glen_b
    Commented Mar 8, 2016 at 23:52

9 Answers 9


One option is to exclude outliers, but IMHO that is something you should only do if you can argue (with almost certainty) why such points are invalid (e.g. measurement equipment broke down, measurement method was unreliable for some reason, ...). E.g. in frequency domain measurements, DC is often discarded since many different terms contribute to DC, quite often unrelated to the phenomenon you are trying to observe.

The problem with removing outliers, is that to determine which points are outliers, you need to have a good model of what is or is not "good data". If you are unsure about the model (which factors should be included, what structure does the model have, what are the assumptions of the noise, ...), then you cannot be sure about your outliers. Those outliers might just be samples that are trying to tell you that your model is wrong. In other words: removing outliers will reinforce your (incorrect!) model, instead of allowing you to obtain new insights!

Another option, is to use robust statistics. E.g. the mean and standard deviation are sensitive to outliers, other metrics of "location" and "spread" are more robust. E.g. instead of the mean, use the median. Instead of standard deviation, use inter-quartile range. Instead of standard least-squares regression, you could use robust regression. All those robust methods de-emphasize the outliers in one way or another, but they typically do not remove the outlier data completely (i.e. a good thing).

  • 8
    $\begingroup$ Great answer. Most people don't realize that not every technique is suited for every type of data. Concentrating on the mean for data ridden with outliers is one of the unfortunate results. The more wake-up calls they get, from answers like this, the better for everybody. $\endgroup$
    – rumtscho
    Commented Mar 9, 2016 at 9:59

I do not recommend excluding any outlier in the main analysis (unless you are really positive they are mistaken). You can do it in a sensitivity analysis, though, and compare the results of the two analyses. In science, often you discover new stuff precisely when focusing on such outliers.

To further elaborate, just think about the seminal Fleming's discovery of penicillin, based on the accidental contamination of his experiments with a mold:


Looking at the near past or present, outlier detection is often used to guide innovation in biomedical sciences. See for instance the following articles (with some suitable R codes):




Finally, if you have reasonable grounds to exclude some data, you may do it, preferably in a sensitivity analysis, and not in the primary one. For instance you could exclude all values which are not biologically plausible (such as a temperature of 48 degrees Celsius in a septic patient). Similarly, you could exclude all first and last measurements for any given patient, to minimize movement artifacts. Take notice however that if you do this post-hoc (not based on a pre-specified criteria), this risks amounting to data massaging.

  • 6
    $\begingroup$ Agree, but I find this answer somehow to brief to upvote it. Maybe you could provide a worked example, or show why and how new stuff can be discovered when focusing on outliers? This may be not that obvious at a first sight. $\endgroup$
    – Tim
    Commented Mar 9, 2016 at 9:35

Thought I'd add a cautionary tale about removing outliers:

Remember the problem with the hole in the polar ozone layer? There was a satellite that was put in orbit over the pole specifically to measure ozone concentration. For a few years the post-processed data from the satellite reported that the polar ozone was present at normal levels, even though other sources clearly showed that the ozone was missing. Finally someone went back to check the satellite software. It turned out that someone had written the code to check if the raw measurement was within an expected range about the typical historical level, and to assume that any measurement outside the range was just an instrument 'spike' (i.e an outlier), auto-correcting the value. Fortunately they had also recorded the raw measurements; on checking them they saw that the hole had been reported all along.

  • 15
    $\begingroup$ It would be good to include a reference to the incident: Why hadn't they discovered the phenomenon earlier? Unfortunately, the TOMS data analysis software had been programmed to flag and set aside data points that deviated greatly from expected measurements and so the initial measurements that should have set off alarms were simply overlooked. In short, the TOMS team failed to detect the ozone depletion years earlier because it was much more severe than scientists expected. $\endgroup$
    – Johnny
    Commented Mar 9, 2016 at 2:06
  • 5
    $\begingroup$ This is a great story. and one much repeated, but to me math.uni-augsburg.de/stochastik/pukelsheim/1990c.pdf convincingly identifies it as a myth based on a misunderstanding. Note incidentally that as there are two poles, "the polar ozone layer" needs re-writing. $\endgroup$
    – Nick Cox
    Commented Mar 10, 2016 at 11:45
  • 3
    $\begingroup$ See also the authoritative account Christie. M. 2001. The Ozone Layer A Philosophy of Science Perspective. Cambridge: Cambridge U.P. $\endgroup$
    – Nick Cox
    Commented Mar 10, 2016 at 11:56

'Outlier' is a convenient term for collecting data together that doesn't fit what you expect your process to look like, in order to remove from the analysis.

I would suggest never (caveat later) removing outliers. My background is statistical process control, so often deal with large volumes of automatically generated time-series data which is processed using a run chart / moving box plot / etc. depending on the data and distribution.

The thing with outliers is that they will always provide information about your 'process'. Often what you are thinking of as one process is actually many processes and it is far more complex than you give it credit for.

Using the example in your question, I would suggest there could be a number of 'processes'. there will be variation due to ...

  • samples taken by one conductance device
  • samples taken between conductance devices
  • when the subject removed a probe
  • when the subject moved
  • differences within one subject's skin across their body or between different sampling days (hair, moisture, oil, etc)
  • differences between subjects
  • the training of the person taking the measurements and variations between staff

All of these processes will produce extra variation in the data and will probably move the mean and change the shape of the distribution. Many of these you won't be able to separate into distinct processes.

So going to the idea of removing data points as 'outliers' ... I would only remove data points, when I can definitely attribute them to a particular 'process' that I want to not include in my analysis. You then need to ensure that the reasons for non inclusion are recorded as part of your analysis, so it is obvious. Don't assume attribution, that's the key thing about taking extra notes through observation during your data collection.

I would challenge your statement 'because most of them are errors anyway', as they are not errors, but just part of a different process that you have identified within your measurements as being different.

In your example, I think it is reasonable to exclude data points that you can attribute to a separate process that you don't want to analyse.


If you are removing outliers the, in most situations you need to document that you're doing so and why. If this is for a scientific paper, or for regulatory purposes, this could result in having your final statistics discounted and/or rejected.

The better solution is to identify when you think you're getting bad data (e.g. when people pull wires), then identify when people are pulling wires, and pull the data for that reason. This will probably also result in some 'good' data points being dropped, but you now have a 'real' reason to tag and discount those data points at the collection end rather than at the analysis end. As long as you do that cleanly and transparently, it's far more likely to be acceptable to third parties. If you remove data points related to pulled wires, and you still get outliers, then the probable conclusion is that the pulled wires are not the (only) problem -- the further problem could be with your experiment design, or your theory.

One of the first experiments my mom had when returning to university to finish her BSc was one where students were given a 'bad' theory about how a process worked, and then told to run an experiment. Students who deleted or modified the resulting 'bad' data points failed the assignment. Those who correctly reported that their data was in disagreement with the results predicted by (the bad) theory, passed. The point of the assignment was to teach students not to 'fix' (falsify) their data when it wasn't what was expected.

Summary: if you're generating bad data, then fix your experiment (or your theory), not the data.

  • 1
    $\begingroup$ That "bad theory" exercise is a great idea! $\endgroup$
    – A. Donda
    Commented Feb 26, 2020 at 16:28

It's a moral dilemma for sure. On one hand, why should you let a few suspicious data points ruin your model's fit to the bulk of the data? On the other hand, deleting observations that don't agree with your model's concept of reality is a censorship of sorts. To @Egon's point, those outliers could be trying to tell you something about that reality.

In a presentation from statistician Steve MacEachern, he defined outliers as being "[not representative of the phenomenon under study.]" Under that viewpoint, if you feel that these suspicious data points are not representative of the skin conductance phenomenon you are trying to study, maybe they don't belong in the analysis. Or if they're allowed to stay, a method should be used that limits their influence. In that same presentation MacEachern gave examples of robust methods, and I remember that, in those few examples, the classical methods with the outliers removed always agreed with the robust analyses with the outliers still included. Personally, I tend to work with the classical techniques I'm most comfortable with and live with the moral uncertainty of outlier deletion.

  • 11
    $\begingroup$ In Box, Hunter & Hunter: "Statistics for Experimenters" they tell that, in the chemical industry, outliers often have led to new patents. Do you want to throw out your new patent? $\endgroup$ Commented Mar 8, 2016 at 20:40
  • 3
    $\begingroup$ Nope, I don't want to miss out on any patents. But I also don't want to spin twelve cycles trying to get my model to accommodate "somebody pulling on the wires." That's almost certainly not the phenomenon under study. I do like the idea of outliers as opportunities, and one thing to be said for straightforward deletion is that at least the code will provide documentation of those deletions, whereas in robust methods the outliers just kind of coexist with the other points. $\endgroup$
    – Ben Ogorek
    Commented Mar 9, 2016 at 14:26
  • 3
    $\begingroup$ You are right that the specific circumstances must be taken into consideration. What should not be done is apply some context-free "rules" for outlier rejection. There do not exist any such good rules. $\endgroup$ Commented Mar 9, 2016 at 14:36
  • 1
    $\begingroup$ My favorite point about the power of context is illustrated by the question, "Are Snickers bars healthy?" Well, if you've been lost in the woods for three days and you've just found a few on the ground, it turns out they're pretty healthy after all. I feel like the popular answers here are telling us, "Never eat a Snickers bar, unless you're absolutely sure you'll die if you don't." $\endgroup$
    – Ben Ogorek
    Commented Mar 13, 2016 at 21:07

My answer aligns with the majority: Do not remove outliers unless you are certain they are erroneous. What I add is:

  • A brief overview of published papers on this topic (those that I am aware of and primarily those published in psychology. There are many more).
  • Based on that, an answer to the question: What method can be used instead of removing outliers when one knows that there are many incorrect data points but not which ones are incorrect?


It is well-documented that the removal of outliers invalidates statistical results. Wilcox 1998 states: "This approach fails [removing outliers before a standard analysis] because it results in using the wrong standard error. Briefly, if extreme values are thrown out, the remaining observations are no longer independent, so conventional methods for deriving expressions for standard errors no longer apply." For a more detailed explanation, see the paper. Bakker et al. (2014) demonstrated one of the effects of this: substantially inflated type I error rates. Recently, Andre (2022) argued that this is only a problem when the model/hypothesis is considered for removing outliers. To provide a concrete example: He stated that while removing outliers within a group is problematic due to the invalid standard errors, removing outliers across groups is valid. More recently, Karch (2023)(disclaimer: that's me) demonstrated that removing outliers across groups is equally problematic: Among other things, if there are group differences, it almost always invalidates confidence intervals and parameter estimates.

What can be used with noisy data?

All the papers cited so far recommend robust methods for handling noisy data (as suggested in some answers). Importantly, contrary to what is claimed in other answers, robust methods do not always yield the same results as outlier removal + standard methods. Briefly, robust methods use the correct standard errors, while outlier removal + standard methods do not (refer to Wilcox for details).

For the situation the original question asks about (comparing two groups), either the Yuen-Welch test or the Brunner-Munzel test and their corresponding confidence intervals seem like they could be applicable. The Yuen-Welch test is essentially the robust version of Welch's t-test. It's important to note that it considers trimmed means instead of normal means, which can be very different for asymmetric distributions (see example by AdamO). Brunner-Munzel's test is essentially the robust alternative to the Wilcoxon-Mann-Whitney test (see https://stats.stackexchange.com/a/579604/30495). Both tests are readily available in R (see https://rdrr.io/cran/WRS2/man/yuen.html, and https://cran.r-project.org/web/packages/brunnermunzel/index.html)

  • 2
    $\begingroup$ +1. But Wilcox has a truly idiosyncratic sense of what "independent" samples are! $\endgroup$
    – whuber
    Commented May 11, 2023 at 21:29

If I conduct a random sample of 100 people, and one of those people happens to be Bill Gates, then as far as I can tell, Bill Gates is representative of 1/100th of the population.

A trimmed mean tells me the average lottery earnings is $0.

  • 1
    $\begingroup$ Nothing abnormal, a trimmed mean is not suitable for skewed distributions. $\endgroup$ Commented Jan 11, 2018 at 17:28

Of course you should remove the outliers, as by definition they do not follow the distribution under scrutiny and are a parasitic phenomenon.

The real question is "how can I reliably detect the outliers" !

  • 1
    $\begingroup$ What if such a distribution is Cauchy? $\endgroup$
    – AdamO
    Commented Jan 11, 2018 at 17:18
  • $\begingroup$ @AdamO: the real question remains, of course. $\endgroup$ Commented Jan 11, 2018 at 17:19
  • $\begingroup$ Why this downvote ? $\endgroup$ Commented Jan 11, 2018 at 17:19
  • 5
    $\begingroup$ (-1) because I don't think this is an adequate contribution informed by theory, example, or practice. What is a "parasitic phenomenon" but a poetic understanding of data? In dealing with blood pressures, urinary sodiums, and neurologic imaging, I see "outliers" on a day-to-day basis which are representative of the population under consideration. Removing them can be a significant source of bias. To say they are a "parasitic phenomenon" is suggestively and deceptively enabling a risky statistical practice. $\endgroup$
    – AdamO
    Commented Jan 11, 2018 at 17:21
  • $\begingroup$ @adam: you are just advocating to keep the inliers, which I fully agree with. $\endgroup$ Commented Jan 11, 2018 at 17:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.