# Estimating the distribution over number of clusters in clustering

I'm reading Murphy's Machine Learning- A Probabilistic Perspective. The introductory section on discovering clusters says:

It seems that there might be various clusters, or subgroups, although it is not clear how many. Let $K$ denote the number of clusters. Our first goal is to estimate the distribution over the number of clusters, $p(K|D)$; this tells us if there are subpopulations within the data. For simplicity, we often approximate the distribution $p(K|D)$ by its mode, $K^∗ = \arg \max_K p(K|D)$.

($D = \{\textbf{x}_i\}_{i=1}^N$ is the training data set)

Earlier, it's stated that in unsupervised learning, we're essentially performing density estimation, i.e. we want to build models of the form $p(\textbf{x}_i | \theta)$.

So I'm confused by the above quoted paragraph- instead of estimating the probability distribution of the feature vector, we seem to be estimating the most likely value of the hyperparameter $K$, which is completely different. Is the $K$ estimation a preliminary step before we get to estimate $p(\textbf{x}_i | \theta)$?And what is exactly meant by "estimate the distribution over the number of clusters"? Distribution of what?

• The number of clusters is yet another random variable that can be estimated. In purely Bayesian setting, you can put a prior on it. The data (and the model + assumptions you choose) will fit better some cluster numbers than others. Over the range of number of clusters (1,...), you will get a probability distribution over clusters. Something similar can be done is sequence segmentation - position where the switch point is not certain: you get a distribution over positions where the switchpoint is likely to occur. I have implemented examples in Infer.NET framework. – Vladislavs Dovgalecs Dec 19 '17 at 19:35

• But roughly, a few Bayesian approaches try to model $p(\vec{x}| \theta_K)$, the density if I would know the correct value of K. For all K. Then choose that which has the best fit, following the usual Bayesian maximum likelihood principle. – Has QUIT--Anony-Mousse Dec 19 '17 at 19:33