6
$\begingroup$

I want to test few similarity measures for outlier detection. I've got some data from UCI repository, for example: Breast-Cancer.

Is there a smart way to add artificial outliers to an existing data?

Thank you.

$\endgroup$
4
  • 1
    $\begingroup$ outliers will be relative to some model/assumption. Observations that are outliers under one model may be perfectly reasonable observations under another. $\endgroup$
    – Glen_b
    Oct 6, 2012 at 11:22
  • $\begingroup$ If I understand you correctly, this can not be done if we don't know statistical properties of the data? $\endgroup$
    – Uros K
    Oct 6, 2012 at 11:27
  • 1
    $\begingroup$ @genesiss: I think the idea is you need to know the statistical properties assumed by your model. Your model will be reflected in the similarity measures you use. $\endgroup$
    – Wayne
    Oct 6, 2012 at 12:31
  • 1
    $\begingroup$ @genesiss Wayne is right; it's the relationship of the model to the data that makes an outlier an outlier. Data in and of itself is just data; it can only be discrepant with regard to some model making it so. $\endgroup$
    – Glen_b
    Oct 7, 2012 at 23:29

3 Answers 3

6
$\begingroup$

You could add random noise to the existing data objects, i.e. changing a given percentage of the data entries to random values within the data range, or swapping some entries between two data objects (which won't change the value distribution in this dimension). This method is often used to test the robustness of algorithms. It could be useful in your case, too.

The method is described here: Assessing data mining results via swap randomization.

Maybe this paper is useful: A synthetic data generator for clustering and outlier analysis

$\endgroup$
2
  • 1
    $\begingroup$ In some sense, they might want to model the outliers. What are the outliers: measurement noise (add zero-mean random value), equipment failure (value will be outside of reasonable data range), recording error (swapped values, swapped digits, digit added or deleted), etc? $\endgroup$
    – Wayne
    Oct 6, 2012 at 12:51
  • $\begingroup$ @Wayne Yes, I agree. My answer is by far not a complete overview over all methods. It's just a collection of methods I used so far. $\endgroup$
    – user14071
    Oct 6, 2012 at 13:24
6
$\begingroup$

There are two commonly seen approaches:

  1. Add outliers to real data by randomization methods.

  2. In order to obtain a rare class, downsample a class to desired sparsity (usually, this should be <<1%)

For 1 there are some variants - modifying single attributes, drawing each attribute, but from different instances etc.; personally, I'm not at all convinced of these methods. Because they simulate a particular effect of data dilution, and thus often favor algorithms designed around the same concept of outlierness. A method that does well on such data sets will then often fail badly when your real outliers are not caused by this very specific kind of errors.

For 2, you will have to face the fact that some data sets are just too hard. The fact that one class is more rare than the others doesn't mean they are really outliers; even if you downsample it to the extreme. Plus, this approach is also quite naive: it assumes that the majority class does not contain outliers. In any real data set that I have seen every class will have outliers within the class, too. So do not expect your method to be able to go to 90% on these data sets. If you can improve from 70% to 80%, then your method already works quite well. Anything beyond 80% may be indicative of some bias IMHO.

When reviewing outlier detection papers, I consider any result higher than 0.80 to be suspicious: either the data set was too much designed for the algorithm, the algorithm parameter were systematically tweaked to find the best possible result, or maybe the result is just fake altogether.

In most cases where I've seen the WBC data set being used, they downsampled the cancer class to like 10 instances. But then, you shouldn't tell your algorithm to get the top 10 results. In a real scenario, you do not know there are 10 outliers to be found...

$\endgroup$
3
  • 2
    $\begingroup$ +1 I fully agree with not being convinced. The bad thing about outliers is not if they are different from your data or don't belong in. The bad thing is that you won't know what causes them and how they look. Otherwise, you'd not call them outliers but give them a name (e.g. in Raman spectroscopy, we have "cosmic ray spikes") and develop a filter for that type of event ("cosmic ray filter"). "Outlier" IMHO is an extremely ill-defined class: the class of unknown things happening to your data that should not happen. $\endgroup$ Nov 16, 2012 at 15:20
  • $\begingroup$ @cbeleites exactly. The whole point about outlier detection is looking for something you do not know what it looks like. But I disagree with you on the term "don't belong in". Very often, outliers do belong to your data. They just somehow "stick out", but they aren't just errors. Just as in your example: the cosmic ray spikes are real; not 'corrupted' data. You do want to know when they occur - although maybe just to skip them. $\endgroup$ Nov 16, 2012 at 17:03
  • 1
    $\begingroup$ I agree, I think I should have said "should not happen, according to your understanding of the data generating process". As long as you cannot say what happened, neither can you know where they belong. Cosmic rays do not belong to Raman spectra but they belong to camera raw data. The default assumption should be that they belong to the data. But I fully agree that "outlier" is far too often an excuse to "shave" the data until it behaves as expected instead of exploring the underlying process. One man's meat is another man's poison: my outlier may be your most important measurement. $\endgroup$ Nov 17, 2012 at 9:42
2
$\begingroup$

Outliers are usually thought of in relation to the model, as the comments already discuss. But that does not say anything about how they are generated: They can be rare events by the very process described by the model (roughly 1 in 10⁹ standard normally distributed numbers will be < -6) or they can be generated by a process that is not included in your model.

Usually, one doesn't care about the former, as the model is adequate for them.

But with respect to the latter, you can simulate only things where you have an idea of the generating process. If you want unexpected rare events, there is no way but collecting data and waiting for them to occur. And IMHO it doesn't make sense to discuss this without discussing the underlying process/problem/task (not only the model). It is the very nature of these things that you cannot give a typical outlier. And you need to discuss the relevance of your outlier generating process to the model and problem. IMHO, the no free lunch theorem applies very much for outlier detection.

Recommended reading ;-) "Journal of Machine Learning Gossip" Paper of which a few copies still float aroud in the net.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.