When evaluating clustering methods which do have a definition for noise points (like dbscan), how noise will affect evaluation?

Consider a clean dataset like well known Iris dataset. There is no noise in it. In evaluation metrics like homogeneity should we remove noise labels and then compute the metric?


True_labels = [1, 1, 1, 2, 2, 1]
Expected_labels = [a, a, -1, b, -1]

Do we need to alter these labels some how like this?

True_labels = [1, 1, 2]
Expected_labels = [a, a, b]

1 Answer 1


There are multiple choices. For example, you could treat all noise as a single cluster, or each object as being its own 1-element cluster.

I don't think any of the evaluation methods works very well in these cases though. Evaluation of clusterings is pretty much broken by design, unfortunately.

  • 2
    $\begingroup$ Try a problem-driven approach. Don't solve math, solve real data problems; don't cluster random data, but try to solve some real world problem. $\endgroup$ Sep 1, 2014 at 13:31
  • 1
    $\begingroup$ what do you plan to do next with the clustering result? $\endgroup$ Sep 1, 2014 at 16:06
  • 1
    $\begingroup$ But what do you do when you have found a cluster? You'll never know if a cluster is good when you stop after clustering. $\endgroup$ Sep 1, 2014 at 17:58
  • 1
    $\begingroup$ Assume your validation was good for now. What do you plan to do next? How do you use the result? $\endgroup$ Sep 1, 2014 at 18:22
  • 1
    $\begingroup$ But it could as well be just one cluster per client IP, then you did not gain anything. I haven't seen any convincing mining on IP data yet... you'll have to try this "sth like that" part, to see if your clustering helps. $\endgroup$ Sep 1, 2014 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.