Outlier vs Anomaly in Machine learning

Question

What is the difference between Outlier and Anomaly in the context of machine learning. My understanding is that both of them refer to the same thing.

Answer 1

The two terms are synonyms according to:

Aggarwal, Charu C. Outlier Analysis. Springer New York, 2017, doi: http://dx.doi.org/10.1007/978-3-319-47578-3_1

Quotation from page 1:

Outliers are also referred to as abnormalities, discordants, deviants, or anomalies in the data mining and statistics literature.

Bold text is not part of the original text.

The free to download pdf of the book available from the author is here.

Answer 2

A tongue-in-cheek answer:

Outlier: a value that you predictably find in your data that indicates your model does not work properly

Anomaly: a value that against all odds you find in your data that indicates your model does work properly

A more serious, less cryptic answer:

The concept of outliers starts from the issue of building a model that makes assumptions about the data. Outliers are often indicators that the model does not describe the data properly and thus we should question the results of our model or quality of our data.

The concept of anomalies starts outside the theoretic world and inside the applied world: we want to look for unusual behavior in our data, sometimes motivated by the fact that we are interested in finding behavior that someone is trying to hide (like a virus in an email). The problem is that since people are trying to hide what they are doing, we don't really know what to look for. So we take a set of "good" data, and decide that whatever we find in our new dataset that doesn't look "good" is an anomaly and worth our time to checkout in more detail. Often, looking for anomalies means looking for outliers in your new data set. But note that these values may be very common in your new dataset, despite being rare in your old dataset!

In summary, the two concepts are very similar in terms of the statistics behind them (i.e. unusual values given your fitted model) but come at the idea from different angles. In addition, when we talk about outliers, we typically mean an unusual data point in the data used to fit our model, where as an anomaly is usually meant as an unusual data point in a dataset outside of the data used to fit our model.

Note: this answer is based on how I've seen the two terms frequently used rather than formal definitions. User experiences may differ.

Answer 3

An anomaly is a result that can't be explained given the base distribution (an impossibility if our assumptions are correct). An outlier is an unlikely event given the base distribution (an improbability).

Answer 4

The terms are largely used in an interchangeable way. "Outlier" refers to something lying outside the norm - so it is "anomalous". But I have the inpression that "outlier" is usually used for very rare observations. In statistics, on a normal distribution, you would consider three sigma to be outliers. That is 99.7% of your objects are expected to be "normal". "Anomaly" is used much more liberally. If you suddenly have millions of visitors on your website, these are not rare visitors. The sudden increase in visitors however is still "anomalous", whereas each individual visitor is not an "outlier".

It may have been in this article where I saw these differences discussed, but I can't access it right now, unfortunately.

Statistical Analysis and Data Mining, Volume 5, Issue 5, October 2012, Pages 363–387 A survey on unsupervised outlier detection in high-dimensional numerical data

Answer 5

Just to muddy the waters further, in climatology anomaly just implies the difference between value and mean, or a deviation:

The term temperature anomaly means a departure from a reference value or long-term average. A positive anomaly indicates that the observed temperature was warmer than the reference value, while a negative anomaly indicates that the observed temperature was cooler than the reference value.

see e.g.

That may well be regarded as outside machine learning, but people interested in the question may be interested in this.

Answer 6

Good question. However, google search on "difference between outliers and anomalies site:.edu" shows that there is no theoretical difference between these two terms. They are being used interchangeably in literature.

Answer 7

An outlier is a data point that makes it hard to fit a model. You face outliers, often unwillingly, when you are trying to fit a model on your dataset. Removing outliers enables building better (i.e. more generalizable) models. A point $(1,5)$ would be an outlier for the model $y=x$. You ignore it in light of the fact that all your other points $(1,1)$, $(5,5)$, $(3,3.1)$ more closely fit $y=x$.

An anomaly can be one data point, or also a general trend or behavior observed in data after a model has already been built or an understanding of the data-generating process formed. You face anomalies because the system starts behaving differently, or you seek out such data points, because you want to be informed when an event occurs during which your model is not valid. You may care about observing any anomalous behavior in amplitudes of ocean waves, not because you want to throw away those data points and build a better model, but because you want to be aware when a tsunami might be happening.

Answer 8

• An outlier is the allowed value which is extremely distanced from the average of other values.

• An anomaly is an unallowed(forbidden) value.

For example, the questionnaire which evaluates a restaurant using the numbers from 1(Bad) to 10(Good) gets the outlier 2 which is extremely distaced from the average 9 and also gets the anomalies -15, -4, 0, 55, A, B and C as shown below:

    　 # Anomaly　# Outlier            # Average    # Anomaly
      ↓↓↓  ↓↓  ↓      ↓                    ↓       ↓↓  ↓  ↓  ↓
      -15  -4  0 | 1  2  3  4  5  6  7  8  9  10 | 55  A  B  C
                 | Bad <<<<<<<<<-->>>>>>>>> Good |
Unallowed values |        Allowed values         | Unallowed values