When evaluating clustering methods which do have a definition for noise points (like dbscan), how noise will affect evaluation?

Consider a clean dataset like well known Iris dataset. There is no noise in it. In evaluation metrics like homogeneity should we remove noise labels and then compute the metric?


True_labels = [1, 1, 1, 2, 2, 1]
Expected_labels = [a, a, -1, b, -1]

Do we need to alter these labels some how like this?

True_labels = [1, 1, 2]
Expected_labels = [a, a, b]

There are multiple choices. For example, you could treat all noise as a single cluster, or each object as being its own 1-element cluster.

I don't think any of the evaluation methods works very well in these cases though. Evaluation of clusterings is pretty much broken by design, unfortunately.

  • $\begingroup$ Clustering illness define and evaluation is breaking my bones at the moment! $\endgroup$ – Mehraban Sep 1 '14 at 13:29
  • 1
    $\begingroup$ Try a problem-driven approach. Don't solve math, solve real data problems; don't cluster random data, but try to solve some real world problem. $\endgroup$ – Has QUIT--Anony-Mousse Sep 1 '14 at 13:31
  • $\begingroup$ Can we set a time to chat about this? $\endgroup$ – Mehraban Sep 1 '14 at 13:46
  • $\begingroup$ I can't help you that much - I don't have a code or formula solution; but the idea is to continue as if your clustering result was good, and then look back after the next step, if you need to go back and retry. $\endgroup$ – Has QUIT--Anony-Mousse Sep 1 '14 at 13:53
  • $\begingroup$ I don't meant to ask for formula or code. I most want to know about the way I need to do the whole thing. $\endgroup$ – Mehraban Sep 1 '14 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.