0
$\begingroup$

Given a dataset X on which I applied k-means and I computed the Silhouette Index score. I consider this score as the truth. I applied again k-means on X and I computed the Silhouette Index score. My question is how to compute the error (%) inducted by the second silhouette score compared to the first silhouette score. I thought to this formulate:

 |score1 - score2|/2.

I divided by 2 because the silhouette score is between -1 and 1.

$\endgroup$
  • $\begingroup$ Same dataset? X and X? Were you using different K-means settings (different number of clusters?) $\endgroup$ – ttnphns Mar 8 at 18:30
  • $\begingroup$ @ttnphns Yes the same dataset X and I apply different K-means settings $\endgroup$ – nabiltos Mar 8 at 18:34
  • 1
    $\begingroup$ You may quantify their difference any way you like. Usually one just plots the values of such criterion for different cluster solutions. Visual comparison is enough. So far I can't understand from your question why you might be wishing more than that. $\endgroup$ – ttnphns Mar 8 at 18:51
  • 1
    $\begingroup$ Well, I think you may go any reasonable formula for you. You should however use, with k-means, not classic Silhouette index but more suited for it "deviation" aka "simplified" version of Silhouette. You may want to read "Clustering criteria" document on my web-page. $\endgroup$ – ttnphns Mar 8 at 19:09
  • 1
    $\begingroup$ The doc. to read is in the "Clustering criterions" archive here spsstools.net/en/macros/KO-spssmacros $\endgroup$ – ttnphns Mar 8 at 19:52
0
$\begingroup$

Averages below 0 shouldn't arise with methods such as k-means, unless there is a severe error. So no need to do a /2. If the method is more than "1" times worse then it failed badly.

But why do you use the abs? That means you lose the information whether you find better or worse results! Please, include the sign!

I don't think it's a good idea to measure the quality of different k-means variations in terms of Silhouette. Just evaluate SSQ, which is the objective used by k-means. So if you make yet another variant (there are literally hundreds of "improvements" of k-means already, are you sure your idea is new?) then it is most appropriate to evaluate how often (and how fast) you find better solutions than the standard k-means++ heuristic with a good algorithm such as Annulus k-means and Exponion k-means. Quality wise, I would give Hartigan and Wong's variant a try, as it may find better solutions than with just Voronoi iterations.

Silhouette is essentially a separate clustering objective. It's similar enough to k-means to make it useful to choose k; at least on small data because of computation cost. But there are also attempts to maximize Silhouette directly or as Post-Processing IIRC. So if you use Silhouette as baseline, you should likely compare to these approaches, too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.