I focus on unsupervised classification i.e. clustering, as suggested by the question.
There is a major difference between supervised and unsupervised classification that needs to be kept in mind. This is that in supervised classification the existing true classes are known, and the problem is defined by the classes available for training. But in many or even most datasets there can be more than one legitimate and potentially meaningful classification. For example, data on people may be classified by gender, country, job, home ownership, party membership, smoker/non-smoker, any conceivable categorical variable really. In supervised classification it is fixed which classification we consider. In clustering it isn't, and which potentially legitimate classification we find is implied by our choice of clustering method and other choices such as variable selection, distance definition, standardisation and transformation. In this sense clustering is constructive, it is not about just passively discovering a unique truth.
Therefore any quality assessment of a clustering should make reference to the aim of clustering, the meaning of the data, and further relevant background information.
Any concept of "overfitting" would be relative to such choices. In fact I believe that the term "overfitting" can be applied to the following phenomenon. Interested in finding reasonably big meaningful data subsets, we may apply a Gaussian mixture model with BIC that in a small enough dataset can give an "intuitively reasonable" clustering, i.e., in line with data visualisation and subject matter knowledge and requirements. If the data set grows, there is more information in the data to discover that any meaningful and potentially well separated subset of the data that is potentially of interest is not exactly Gaussian, and eventually the BIC will decide to fit any such subset not with a single Gaussian, but with a mixture of more than one of these. If now every Gaussian component is interpreted as its own separate cluster, this may be inappropriate, as there may not be any meaningful difference between some clusters, rather only more than one Gaussian was fitted to emulate a non-Gaussian but still homogeneous shape. Note that this is a problem with the interpretation of the mixture rather than with the fit of the data itself.
The same mixture may do a good job as a density estimator without any interpretation of the individual components, but this is not what we are interested in. We may also be interested in a density estimate with a limited number of parameters and the BIC may ultimately tell us we should use too many, i.e., the trade-off between fit of the data and parsimony requirements motivated by the use we want to make of the model may play out differently than formalised in the BIC or potentially other criteria. The ICL for example will estimate fewer components in such a situation but may occasionally leave together what could be meaningfully split up.
Another formal problem with Gaussian mixtures that can be seen as connected with "overfitting" is the potential degeneration of the likelihood if so few observations are put in a cluster that an eigenvalue of the associated covariance matrix is (very close to) zero, assuming sufficiently flexible covariance matrices. Computing a maximum likelihood estimator as done by the EM-algorithm, this problem means that in fact we don't want to find a global optimum (as the likelihood can degenerate to infinity), but rather a good local optimum not too close to degeneration ("spurious clusters"). Also more constrained models (like assuming equal covariance matrices) can avoid this issue, i.e., a too flexible fit is problematic (although flexibility may be required to achieve a good fit with a low enough number of components).
A too flexible fit may also turn out to be problematic in some kind of cross-validation, namely splitting or bootstrapping the dataset, running clustering on both (or more than two) parts, and then checking whether the different clusterings are reasonably similar. If not, there is a stability problem, which may be caused by too flexible models (i.e., some kind of overfitting). More on validation based on data splitting in clustering is here: Ullmann, Hennig, and Boulesteix: Validation of cluster analysis results on validation data: A systematic framework.
In unsupervised learning, lack of supervision means that the data shouldn't be trusted to make all the decisions on their own. There may be over- or underfitting relative to what we need in a given situation.