3
$\begingroup$

Using K-means I have clustered my data into 3 clusters. After further analysis on these clusters I found the following:

  • Cluster 1 - Low
  • Cluster 2 - Medium
  • Cluster 3 - High

Again, I clustered the data within Cluster 3 (High) into 3 clusters.

Now, I was asked how feasible/rational is this analysis? Why not just derive 6 clusters in the first place?

$\endgroup$

1 Answer 1

2
$\begingroup$

The objective of k-means, for a given $k$, is to find a partition $S = S_1 \bigcup \cdots \bigcup S_k$ which is

$$ \arg \min_{S} \sum_{i = 1}^k \sum_{x \in S_i} \left| x - \mu_i \right|^2 . \; (1) $$

In your second, non-hierarchical, case, $k = 6$. In your first, hierarchical case, you first use $k = 3$, then continue until $k = 6$. This means that you are solving

$$ \arg \min_{S; H(S)} \sum_{i = 1}^k \sum_{x \in S_i} \left| x - \mu_i \right|^2 , \; (2) $$

where $H(S)$ is a constraint on the hierarchy of the sets. Since a constraint cannot lower the minimum, (2) cannot be lower than (1). If your k-mean applications actually approximate the minimums, then it is unlikely that (2) will work better than (1) in terms of the minimization objective.

Hierarchical k-means, however, have other advantages, in terms of execution speed. Note that in this paper, the authors explicitly address how to avoid having the constraint impacting the minimum by too much.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.