4
$\begingroup$

I was advised by my team leader to use this weighted average entropy to evaluating the performance of my dbscan clustering algorithm, and its mathematical formulation is:

WAE maths formulae

Scikit provides what many would consider standard metrics for clustering performance evaluation such as Adjusted Mutual Information, Homogeneity, completeness and V-measure (which are all based on the calculation of entropy.

I wonder the difference and relationship between two metics, aka Weighted Average Entropy and Adjusted Mutual Information? And are they both good for clustering evaluation? Thanks!

$\endgroup$
  • $\begingroup$ None is a good measure for evaluating clusterings. Evaluation of clusterings is a black art, and needs lots of manual labor. Looking at some number is not good. $\endgroup$ – Anony-Mousse Sep 25 '14 at 8:48
  • $\begingroup$ If the ground truth labels are not known, evaluation must be performed using the model itself. The Silhouette Coefficient (sklearn.metrics.silhouette_score) is an example of such an evaluation scikit-learn.org/stable/modules/clustering.html $\endgroup$ – KLDavenport May 31 '15 at 16:24
4
$\begingroup$

It looks like that $\mbox{WAE}$ is just a conditional entropy. If $D$ is your class (ground truth clustering) and $C$ is the clustering solution you generated: $$ \mbox{WAE} = H(D|C) = - \sum_{j=1}^c \frac{|c_j|}{n} \sum_{i=1}^cP_{i|j} \log_2{P_{i|j}}$$ (There is one minus missing in your formula)

  • $H(D|C) = 0$ when your clustering $C$ is identical to the ground truth $D$
  • $H(D|C) = H(D)$ when the clustering solution $C$ is "bad", which is when the random variables corresponding to $D$ and $C$ are independent.

The Adjusted Mutual Information ($\mbox{AMI}$) employs the mutual information ($\mbox{MI}$) to compare clusterings: $$ \mbox{MI} = H(D) - H(D|C) = H(D) - \mbox{WAE} $$ $\mbox{MI}$ uses $\mbox{WAE}$ to obtain a similarity measure which has to be maximized: the bigger the better. $$ \mbox{AMI} = \frac{\mbox{MI} - E[\mbox{MI}]}{\max{\mbox{MI}} - E[\mbox{MI}]}$$

$\mbox{AMI}$ is a further rescaling of $\mbox{MI}$ to obtain:

  • A maximum value of $1$ when clusterings are identical;
  • $0$ value when $\mbox{MI}$ is equal to its expected value obtained by random chance agreement between $C$ and $D$.

Given that there are no explicit random variables involved, the baseline value of $0$ for $\mbox{AMI}$ makes more sense.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.