I am currently trying to make a DBSCAN clustering using scikit learn in python. I would like to compare the different outputs when varying the epsilon parameter in order to choose the right epsilon parameter. I took as an example the iris dataset.
In order to compare clusters I thought about trying to cluster with epsilon within a range (ex : 0.1, 0.2, ..., 1). Now, when I run a kmeans or a hierarchical clustering I can choose my k value by checking the gap statistic for example, or by looking at inertia and choosing a k for which there is an 'elbow' on the inertia vs k plot.
My problem is that I assume this will not work anymore because the total number of points within all the clusters is not constant in DBSCAN. Indeed, depending on epsilon, the number of 'noisy sample' unclassified points will vary. As a result, I may have only a few points for a low epsilon, resulting on a very small Inertia which would be biased. I could consider gap statistic because I may be able to generate random samples with the right size each time. But I wonder if I will then leave the validity framework of the paper and I'm not sure the different clustering can still be compared.
Has anyone an idea on how to compare different total sizes clustering, and more precisely the results of dbscan for different epsilon? Would silhouette coefficient work or is it total size sensitive too?