I'm investigating the bias-variance tradeoff in well-known machine learning algorithms. However, I'm not sure this concept applies in the case of unsupervised methods such as clustering algorithms.
Is it safe to say that in clustering, a small number of clusters will lead to a high bias (underfitting) while a large one will provide a high variance (overfitting)?
I think the bias-variance tradeoff has more to do with the results in general and the underlying distribution of the thing you are doing. If you use a model that's simpler than you know it's require, then you are introducing a bias. But if you use a very complex algorithm you will probably end up with a result that can vary a lot in each run or in each version of the data (assuming you could do resampling). In your case, I think this concept makes sense.
To confirm this, you could also test this empirically. But I'm having trouble in thinking how you could do that. You would need to assess how far the result of a clustering algorithm is from the ground-truth result. Also you would need to run the test a number of times and see how the results with the same number of clusters vary. Using K-Means, for example, I don't know how you could compare results with different number of clusters.
As a side note, I'm reading a book called Feature Engineering and Selection: A Practical Approach for Predictive Models and it has a couple of nice examples of this tradeoff. For example, resampling methods (K-Fold CV, bootstrapping, Monte-Carlo sampling etc) also have a bias-variance tradeoff (although they aren't a model). You can measure the resampling variance and bias using the average model metric that's calculated from the different versions of your data set. 10-Fold CV has the advantage of having low bias (since you are using 90% of the data to fit the models), but usually yielding metrics that can vary a lot. On the opposite, bootstrapping has a higher bias (it's a resampling method with replacement) but normally yields results with small variance. Section 3.4.6 of the book has a nice discussion about that.
