I stumbled across this example on scikit-learn (1.2.0), where the silhouette score alongside some other metrics is computed for DBSCAN cluster assignments. These assignments include some Noise assignments.

from sklearn.cluster import DBSCAN
from sklearn.datasets import make_blobs
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

centers = [[1, 1], [-1, -1], [1, -1]]
X, labels_true = make_blobs(n_samples=750, centers=centers, cluster_std=0.4,
X = StandardScaler().fit_transform(X)
db = DBSCAN(eps=0.3, min_samples=10).fit(X)
labels = db.labels_
print(f"Silhouette Coefficient with Noise as one cluster: {silhouette_score(X, labels):.3f}")
# 0.626

I find strange that there is no special treatment for Noise assigments; the cluster algorithms effectively treat them as a one cluster (since it's all -1). Is this correct?

Because if I make them individual clusters instead, I get a very different result:

for idx, val in enumerate(labels):
    if val == -1:
        labels[idx] = -idx

print(f"Silhouette Coefficient with Noise as individual clusters: {silhouette_score(X, labels):.3f}")
# 0.092

Alternatively, one could ignore the Noise assignments altogether, although this may affect comparison with other metrics:

mask_ok = [x >= 0 for x in labels]

print(f"Silhouette Coefficient ignoring Noise: {silhouette_score(X[mask_ok], labels[mask_ok]):.3f}")
# 0.664
  • $\begingroup$ What is the question? If the question is "is it correct to assume that all 'noise' instances are in the same super-cluster" the answer: It isn't a big deal. I would actual say that you are indeed right, as these data should be ignored but as you see yourself the actual difference is not huge. $\endgroup$
    – usεr11852
    Dec 22, 2022 at 16:02
  • $\begingroup$ The difference is substantial either way: 1) Treat each cluster individually: -85%; 2) Drop all Noise assignments: +6%. $\endgroup$
    – MERose
    Jan 23 at 19:21


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.