I'm working on comparing multiple clustering algorithms to each other using the adjusted Rand index for a given dataset. We have a gold standard that we'd like to compare the obtained clustering assignments against. My main question is:
Is it common place to use cross-validation to compare adjusted rand index values?
I can't seem to find a good answer to my question in the literature. The other problem is that for some clustering algorithms I'm using, have really no good way of computing the clustering assignment for data that is removed. I'm thinking of hierarchal clustering and spectral clustering.
In place of running cross-validation, I'm simply rerunning the clustering techniques 10 times and then computing t-test to determine if the difference is statistically significant. My clustering analysis days have me thinking that this is going to be a problem.