Fuzzy K-means - Cluster Sizes

Question

I'm trying to do fuzzy k-means clustering on a dataset using the cmeans function (R) . The problem Im facing is that the sizes of clusters are not as I would like them to be. This is done by calculating the cluster to which the observations are "closest".

cl$size
 [1]   108    31   192    51   722 18460    67  1584   419 17270

Here we see that for 10 clusters we have two huge clusters and a lot of very small ones. Does this imply that two clusters are optimal in any way? If I do regular K-means 10 segments look very well, with good sizes and their intepretation makes a lot of sense but I would like to try fuzzy correctly. I just started exploring this fuzzy clustering so any help and pointers are overly welcome.

Answer 1

K-means and also fuzzy k-means (emphasized by your "the winner takes it all" strategy) assume that clusters have the same spatial extend.

This is best explained by looking at an object $o$ almost half-way between cluster centers $c_i$ and $c_j$. If it is slightly closer to $c_i$, it will go into cluster $i$, if it is slightly closer to $c_j$ it will go into $j$. I.e. k-means assumes that splitting the dataset at the hyperplane orthogonal on the mean between the two points (i.e. the Voronoi cell boundary) is the proper split and does not at all take into account that clusters may have different spatial extend.

EM clustering (Wikipedia) with Gaussian mixtures is essentially an extension of Fuzzy k-means that does a) not assume all dimensions are equally important and b) the clusters may have a different spatial extend. You could simplify it by removing the Gaussian Mixture (or at least the covariances), and just keep a cluster weight. This weight would essentially give the relative cluster size.

Or you might want to look into more advanced methods for arbitrary shaped clusters such as DBSCAN (Wikipedia) and OPTICS (Wikipedia).

Answer 2

Why not using fanny() from the cluster package? You can find the details of the algorithm in the details section of the help.

Here is the example code from ?fanny

## generate 10+15 objects in two clusters, plus 3 objects lying
## between those clusters.
x <- rbind(cbind(rnorm(10, 0, 0.5), rnorm(10, 0, 0.5)),
           cbind(rnorm(15, 5, 0.5), rnorm(15, 5, 0.5)),
           cbind(rnorm( 3,3.2,0.5), rnorm( 3,3.2,0.5)))
fannyx <- fanny(x, 2)
## Note that observations 26:28 are "fuzzy" (closer to # 2):
fannyx
summary(fannyx)
plot(fannyx)

(fan.x.15 <- fanny(x, 2, memb.exp = 1.5)) # 'crispier' for obs. 26:28
(fanny(x, 2, memb.exp = 3))               # more fuzzy in general

data(ruspini)
f4 <- fanny(ruspini, 4)
stopifnot(rle(f4$clustering)$lengths == c(20,23,17,15))
plot(f4, which = 1)
## Plot similar to Figure 6 in Stryuf et al (1996)
plot(fanny(ruspini, 5))