How to add outliers to an existing data?

Question

I want to test few similarity measures for outlier detection. I've got some data from UCI repository, for example: Breast-Cancer.

Is there a smart way to add artificial outliers to an existing data?

Thank you.

Answer 1

You could add random noise to the existing data objects, i.e. changing a given percentage of the data entries to random values within the data range, or swapping some entries between two data objects (which won't change the value distribution in this dimension). This method is often used to test the robustness of algorithms. It could be useful in your case, too.

The method is described here: Assessing data mining results via swap randomization.

Maybe this paper is useful: A synthetic data generator for clustering and outlier analysis

Answer 2

There are two commonly seen approaches:

1. Add outliers to real data by randomization methods.

2. In order to obtain a rare class, downsample a class to desired sparsity (usually, this should be <<1%)

For 1 there are some variants - modifying single attributes, drawing each attribute, but from different instances etc.; personally, I'm not at all convinced of these methods. Because they simulate a particular effect of data dilution, and thus often favor algorithms designed around the same concept of outlierness. A method that does well on such data sets will then often fail badly when your real outliers are not caused by this very specific kind of errors.

For 2, you will have to face the fact that some data sets are just too hard. The fact that one class is more rare than the others doesn't mean they are really outliers; even if you downsample it to the extreme. Plus, this approach is also quite naive: it assumes that the majority class does not contain outliers. In any real data set that I have seen every class will have outliers within the class, too. So do not expect your method to be able to go to 90% on these data sets. If you can improve from 70% to 80%, then your method already works quite well. Anything beyond 80% may be indicative of some bias IMHO.

When reviewing outlier detection papers, I consider any result higher than 0.80 to be suspicious: either the data set was too much designed for the algorithm, the algorithm parameter were systematically tweaked to find the best possible result, or maybe the result is just fake altogether.

In most cases where I've seen the WBC data set being used, they downsampled the cancer class to like 10 instances. But then, you shouldn't tell your algorithm to get the top 10 results. In a real scenario, you do not know there are 10 outliers to be found...

Answer 3

Outliers are usually thought of in relation to the model, as the comments already discuss. But that does not say anything about how they are generated: They can be rare events by the very process described by the model (roughly 1 in 10⁹ standard normally distributed numbers will be < -6) or they can be generated by a process that is not included in your model.

Usually, one doesn't care about the former, as the model is adequate for them.

But with respect to the latter, you can simulate only things where you have an idea of the generating process. If you want unexpected rare events, there is no way but collecting data and waiting for them to occur. And IMHO it doesn't make sense to discuss this without discussing the underlying process/problem/task (not only the model). It is the very nature of these things that you cannot give a typical outlier. And you need to discuss the relevance of your outlier generating process to the model and problem. IMHO, the no free lunch theorem applies very much for outlier detection.

Recommended reading ;-) "Journal of Machine Learning Gossip" Paper of which a few copies still float aroud in the net.