Clustering Analysis for large data in R

Question

I am trying to perform a clustering analysis for a csv file with 50k+ rows, 10 columns. I tried k-mean, hierarchical and model based clustering methods. Only k-mean works because of the large data set. However, k-mean does not show obvious differentiations between clusters. So I am wondering is there any other way to better perform clustering analysis?

The data looks like this

Revenue  Employee  Longitude Latitude  LocalEmployee BooleanQuestions ...
1000     100       xxxx      xxxx      10
...                                                                   ...

Here is part of my code:

mydata <- scale(mydata)
wss <- (nrow(mydata)-1)*sum(apply(mydata,2,var))
for(i in 2:15)wss[i]<- sum(fit=kmeans(mydata,centers=i,15)$withinss)
plot(1:15,wss,type="b",main="15 clusters",xlab="no. of cluster",ylab="with clsuter sum of squares")

fit <- kmeans(mydata,7)
clusplot(mydata, fit$cluster, color=TRUE, shade=TRUE, labels=2, lines=0)

Answer 1

Understand the algorithms you are using.

Don't treat them as black boxes!

k-means, for example, is known

1. to be sensitive to outliers
2. to not work well with correlations
3. to not work well with binary variables
4. to minimize within cluster variance

so I'm not surprised the result looks like this. You may need to more precisely figure out what your objective is. Maybe variance itself is not what you need to minimize.