# Clustering a noisy data or with outliers

I have a noisy data of two variables like this.

x1 <- rep(seq(0,1, 0.1), each = 3000)
set.seed(123)
y1 <- rep (c(0.2, 0.8, 0.3, 0.9, 0.65, 0.35,0.7,0.1,0.25, 0.3, 0.95), each = 3000)
set.seed(1234)
e1 = rnorm(length(x1), 0.07,0.07)
set.seed(1223)
e2 = rnorm(length(x1), 0.07,0.07)
set.seed(1334)
yn <- rnorm(20000, 0.5,0.9)
set.seed(2344)
xn <- rnorm(20000, 0.5,0.9)
y <- c(y1 + e1,yn)
x <- c(x1 + e2, xn)
plot(x,y,  xlim=c(0,1.2), ylim = c(0,1.2), pch = ".", col = "gray40")


I can visually see there are potential 10 clusters in closer look.

However the whole data has much points spread:

plot(x,y,   pch = ".", col = "gray40")


I would like to make 10 clusters. I tried K-means cluster analysis.

xm1 <- cbind(x,y)
cl1 <- kmeans(xm1, 10)
colrs <- c("red", "green", "blue1", "pink", "green4","tan",
"gray40", "yellow", "black", "purple")
plot(xm1, col = colrs[cl1$cluster], pch = ".", xlim=c(0,1.2), ylim = c(0,1.2))  plot(xm1, col = colrs[cl1$cluster], pch = ".")


Is there anyway (may be kernel k-means, nearest neighbors) that can do more justice to this type of data. If so how can I do this ?

-
What's wrong with what you have so far? Why is what you have (the k-means cluster analysis) unacceptable? –  Steve S Aug 3 '14 at 3:41
Have a look to dbscan or optics algorithms (see en.wikibooks.org/wiki/Data_Mining_Algorithms_In_R/Clustering/…) –  Giorgio Spedicato Aug 3 '14 at 11:20

## 2 Answers

As your data seems to be composed of Gaussian Mixtures, try Gaussian Mixture Modeling (aka: EM clustering). This should yield results far superior to k-means on this type of data.

If your "noise" is uniform distributed, you can also add a uniform distribution to your mixture model.

If your data is much less clean, consider using DBSCAN, MeanShift, OPTICS, HDBSCAN*, ... - density based clusterig seems to be appropriate for this data. DBSCAN is also very tolerant to noise (the "N" is for noise).

-

I recommend you to look at this article. The authors propose robust method where the outliers are removed and the rest of data is clustered. That is why they called the method "trimming". There was also an R package tclust but according to this, it was removed from CRAN. Anyway, the article is worth reading.

-