Context --
Unlike, Partitional clustering algorithms like K-Means, Spectral or Hierarchal Methods, Incomplete clustering techniques like DBSCAN, HDBSCAN and many others have the notion of noise (outliers).
Common cluster validation (Internal and External) indices - Silhouette Index, C-Index, Dunn Index, Entropy, Within-cluster scatter ... don't explicitly accommodate noise.
Questions --
- Do there exist special Cluster Validation indices that accommodate noise?
- What would be the appropriate treatment of noise, so that they fit well into all cluster indices ?
The cluster validation should reflect the loss of cluster-ability due to excessive noise.
Some thoughts --
- Treat the noise as a new (k+1 th) partition.
- Remove all noises from the picture (Not recommended)
- Re-assign noises into one of the existing clusters (by nearest neighbor technique or using some cluster indices)
Please feel free to put forward your suggestions and links to related research materials and posts.
Edit : Let's assume a two class problem.