10
$\begingroup$

I'm having difficulty understanding one or two aspects of the cluster package. I'm following the example from Quick-R closely, but don't understand one or two aspects of the analysis. I've included the code that I am using for this particular example.

## Libraries
library(stats)
library(fpc) 

## Data
mydata = structure(list(a = c(461.4210925, 1549.524107, 936.42856, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 131.4349206, 0, 762.6110846, 
3837.850406), b = c(19578.64174, 2233.308842, 4714.514274, 0, 
2760.510002, 1225.392118, 3706.428246, 2693.353714, 2674.126613, 
592.7384164, 1820.976961, 1318.654162, 1075.854792, 1211.248996, 
1851.363623, 3245.540062, 1711.817955, 2127.285272, 2186.671242
), c = c(1101.899095, 3.166506463, 0, 0, 0, 1130.890295, 0, 654.5054857, 
100.9491289, 0, 0, 0, 0, 0, 789.091922, 0, 0, 0, 0), d = c(33184.53871, 
11777.47447, 15961.71874, 10951.32402, 12840.14983, 13305.26424, 
12193.16597, 14873.26461, 11129.10269, 11642.93146, 9684.238583, 
15946.48195, 11025.08607, 11686.32213, 10608.82649, 8635.844964, 
10837.96219, 10772.53223, 14844.76478), e = c(13252.50358, 2509.5037, 
1418.364947, 2217.952853, 166.92007, 3585.488983, 1776.410835, 
3445.14319, 1675.722506, 1902.396338, 945.5376228, 1205.456943, 
2048.880329, 2883.497101, 1253.020175, 1507.442736, 0, 1686.548559, 
5662.704559), f = c(44.24828759, 0, 485.9617601, 372.108855, 
0, 509.4916263, 0, 0, 0, 212.9541122, 80.62920455, 0, 0, 30.16525587, 
135.0501384, 68.38023073, 0, 21.9317122, 65.09052886), g = c(415.8909649, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 637.2629479, 0, 0, 
0), h = c(583.2213618, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0), i = c(68206.47387, 18072.97762, 23516.98828, 
13541.38572, 15767.5799, 19756.52726, 17676.00505, 21666.267, 
15579.90094, 14351.02033, 12531.38237, 18470.59306, 14149.82119, 
15811.23348, 14637.35235, 13588.64291, 12549.78014, 15370.90886, 
26597.08152)), .Names = c("a", "b", "c", "d", "e", "f", "g", 
"h", "i"), row.names = c(NA, -19L), class = "data.frame")

Then I standardize the variables:

# standardize variables
mydata <- scale(mydata) 

## K-means Clustering 

# Determine number of clusters
wss <- (nrow(mydata)-1)*sum(apply(mydata,2,var))
for (i in 2:15) wss[i] <- sum(kmeans(mydata, centers=i)$withinss)
# Q1
plot(1:15, wss, type="b", xlab="Number of Clusters",  ylab="Within groups sum of squares") 

# K-Means Cluster Analysis
fit <- kmeans(mydata, 3) # number of values in cluster solution

# get cluster means 
aggregate(mydata,by=list(fit$cluster),FUN=mean)

# append cluster assignment
mydata <- data.frame(mydata, cluster = fit$cluster)

# Cluster Plot against 1st 2 principal components - vary parameters for most readable graph
clusplot(mydata, fit$cluster, color=TRUE, shade=TRUE, labels=0, lines=0) # Q2

# Centroid Plot against 1st 2 discriminant functions
plotcluster(mydata, fit$cluster)

My question is, how can the plot which shows the number of clusters (marked Q1 in my code) be related to the actual values (cluster number and variable name) ?

Update: I now understand that the clusplot() function is a bivariate plot, with PCA1 and PCA2. However, I don't understand the link between the PCA components and the cluster groups. What is the relationship between the PCA values and the clustering groups? I've read elsewhere about the link between kmeans and PCA, but I still don't understand how they can be displayed on the same bivariate graph.

Improve this question
$\endgroup$
3
  • $\begingroup$ apologies if I've asked too many questions about clustering over the last few days. I'm trying to become more familiar with this field quickly (also I did post this question on SO, stackoverflow.com/questions/4997870/… but it was suggested to move it here) $\endgroup$
    – celenius
    Feb 15, 2011 at 16:03
  • $\begingroup$ That's ok, this is not TCS.SE (-; $\endgroup$
    – user88
    Feb 15, 2011 at 16:47
  • $\begingroup$ # Determine number of clusters Could you explain why we are using this formula (mydata,2,var) and why 2:15? $\endgroup$
    – user105300
    Feb 16, 2016 at 8:46

1 Answer 1

Reset to default
8
$\begingroup$

I did not grasp question 1 completely, but I'll attempt an answer. The plot of Q1 shows how the within sum of squares (wss) changes as cluster number changes. In this kind of plots you must look for the kinks in the graph, a kink at 5 indicates that it is a good idea to use 5 clusters.

WSS has a relationship with your variables in the following sense, the formula for WSS is

$\sum_{j} \sum_{x_i \in C_j} ||x_i - \mu_j||^2$

where $\mu_j$ is the mean point for cluster $j$ and $x_i$ is the $i$-th observation. We denote cluster j as $C_j$. WSS is sometimes interpreted as "how similar are the points inside of each cluster". This similarity refers to the variables.

The answer to question 2 is this. What you are actually watching in the clusplot() is the plot of your observations in the principal plane. What this function is doing is calculating the principal component score for each of your observations, plotting those scores and coloring by cluster.

Principal component analysis (PCA) is a dimension reduction technique; it "summarizes" the information of all variables into a couple of "new" variables called components. Each component is responsible of explaining certain percentage of the total variability. In the example you read "This two components explain 73.95% of the total variability".

The function clusplot() is used to identify the effectiveness of clustering. In case you have a successful clustering you will see that clusters are clearly separated in the principal plane. On the other hand, you will see the clusters merged in the principal plane when clustering is unsuccessful.

For further reference on principal component analysis you may read wiki. if you want a book I suggest Modern Multivariate Techniques by Izenmann, there you will find PCA and k-means.

Hope this helps :)

Improve this answer
$\endgroup$
11
  • 1
    $\begingroup$ Thank you for your answer. I had one further question about the clusplot() function. What is the relationship between the PCA values, and the clustering groups? I've read elsewhere about the link between kmeans and PCA, but I still don't understand how they can be displayed on the same bivariate graph. (Perhaps this should be a new question in itself). $\endgroup$
    – celenius
    Feb 22, 2011 at 18:32
  • 2
    $\begingroup$ Actually, the PCA values and the clustering groups are independent. PCA creates "new" coordinates for each observation in mydata, that is what you actually see on the plot. The shape of the points is plotted using fit$cluster, the second parameter of clusplot(). Maybe you should take a deeper look into PCA. Let me know if this helped you, or if you further references. $\endgroup$
    – deps_stats
    Feb 22, 2011 at 21:01
  • 1
    $\begingroup$ It helps (in the sense that I am honing in on my problem!). How is fit$cluster related to the PCA "coordinates"? I think I understand how PCA works, but as I understand it, each Component cannot be explained using variables from the original data (rather it is a linear combination of the raw data), which is why I don't understand how it can be related to the clusters. $\endgroup$
    – celenius
    Feb 22, 2011 at 22:15
  • 2
    $\begingroup$ You almost got it :) fit$cluster is unrelated to PCA. What clusplot() does is to plot the points using the "new" coordinates and label them using fit$cluster. I got '+' for cluster 3, 'o' for cluster 1 and a triangle for cluster 2. The function clusplot() is useful to visualize the clustering. $\endgroup$
    – deps_stats
    Feb 23, 2011 at 15:52
  • 1
    $\begingroup$ By the "new" coordinates I mean PCA1 and PCA2. You're right, they are completely unrelated to fit$cluster :) $\endgroup$
    – deps_stats
    Feb 23, 2011 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.