Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a noisy data of two variables like this.

x1 <- rep(seq(0,1, 0.1), each = 3000)
y1 <- rep (c(0.2, 0.8, 0.3, 0.9, 0.65, 0.35,0.7,0.1,0.25, 0.3, 0.95), each = 3000)
e1 = rnorm(length(x1), 0.07,0.07)
e2 = rnorm(length(x1), 0.07,0.07)
yn <- rnorm(20000, 0.5,0.9)
xn <- rnorm(20000, 0.5,0.9)
y <- c(y1 + e1,yn) 
x <- c(x1 + e2, xn) 
plot(x,y,  xlim=c(0,1.2), ylim = c(0,1.2), pch = ".", col = "gray40") 

I can visually see there are potential 10 clusters in closer look.

enter image description here

However the whole data has much points spread:

plot(x,y,   pch = ".", col = "gray40") 

enter image description here

I would like to make 10 clusters. I tried K-means cluster analysis.

xm1 <- cbind(x,y)
cl1 <- kmeans(xm1, 10)
colrs <- c("red", "green", "blue1", "pink", "green4","tan", 
 "gray40", "yellow", "black", "purple") 
plot(xm1, col = colrs[cl1$cluster], pch = ".", xlim=c(0,1.2), ylim = c(0,1.2))

enter image description here

plot(xm1, col = colrs[cl1$cluster], pch = ".")

enter image description here

Is there anyway (may be kernel k-means, nearest neighbors) that can do more justice to this type of data. If so how can I do this ?

share|improve this question
What's wrong with what you have so far? Why is what you have (the k-means cluster analysis) unacceptable? – Steve S Aug 3 '14 at 3:41
Have a look to dbscan or optics algorithms (see…) – Giorgio Spedicato Aug 3 '14 at 11:20
up vote 6 down vote accepted

As your data seems to be composed of Gaussian Mixtures, try Gaussian Mixture Modeling (aka: EM clustering). This should yield results far superior to k-means on this type of data.

If your "noise" is uniform distributed, you can also add a uniform distribution to your mixture model.

If your data is much less clean, consider using DBSCAN, MeanShift, OPTICS, HDBSCAN*, ... - density based clusterig seems to be appropriate for this data. DBSCAN is also very tolerant to noise (the "N" is for noise).

share|improve this answer

I recommend you to look at this article. The authors propose robust method where the outliers are removed and the rest of data is clustered. That is why they called the method "trimming". There was also an R package tclust but according to this, it was removed from CRAN. Anyway, the article is worth reading.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.