# Mixture distributions: an intuition on why we cannot infer the number of mixture components by visual inspection

I am studying mixture models and I would like your help with this question:

Consider the distribution $$\Gamma$$ and assume it is a finite mixture distribution, i.e., $$\Gamma=\sum_{k=1}^K \Gamma_k \lambda_k$$ with each $$\lambda_k\geq 0$$. Can you an intuition on why we cannot infer the number of mixture components, $$K$$, (or an upper bound on it) just by visual inspection (e.g., by counting how many "bells" we see in the graphical representation of the density function)?

• The mixture$$\sum_{i=1}^K \frac{1}{K} \mathcal N(0,\frac{1}{i^2})$$has $K$ components and a single mode. Commented Apr 30 at 8:12
• All continuous distributions are finite mixtures with arbitrarily large $K.$ This can be proven through explicit construction, as shown at stats.stackexchange.com/a/486951/919 and stats.stackexchange.com/a/299765/919 for two ways to accomplish this.
– whuber
Commented Apr 30 at 13:45
• Thanks. I agree with that (see also here mathoverflow.net/questions/397076/about-a-mixture ) However, my question is about why we cannot infer $K$ just by looking at the graph of the pdf.
– Star
Commented Apr 30 at 13:50
• Well, you can certainly "infer" the number of mixture components by visual inspection. The thing is, your inference may not be any good. (Bad estimators are also estimators. ;-) Commented Apr 30 at 23:08
• If you believe my assertion that $K$ can be arbitrarily large, then there's nothing one can infer about $K$, is there?
– whuber
Commented May 1 at 13:20

## 2 Answers

Firstly, it is not always the case that we will see distinct "bells" that can be associated with specific components - we may see a single bell the appears too wide or having a shoulder... an individual judgement, depending on who looks, on the aspect ration of the graph, etc. So our guess about the number of components may be wrong.

Secondly, not every distribution is amenable to a visual inspection. This surely works for a mixture of one dimensional Gaussian, but what about a distribution with hundreds of features, and possibly non-Gaussian? One example is the gut microbial data, which are known to cluster into several groups ("enterotypes"): we typically have hundreds of microbial species for every individual, which can be modeled and classified using Dirichlet Multinomial Distribution.

Finally, we often need a task to be done automatically, without involving human assistance. So we need a clear algorithm, translatable in a computer code, to perform such a task.

It's helpful here to think of counter examples.

Here's a histogram of an equal mixture of two Normal distributions with different standard deviations but similar means,

m1 = np.random.normal(0, 1, size=10000)
m2 = np.random.normal(0, 5, size=10000)
_ = plt.hist(0.5*m1 + 0.5*m2, bins=40)


Here's a histogram where one mixture is much smaller than the other,

m1 = np.random.normal(1, 1, size=10000)
m2 = np.random.normal(0, 1, size=10000)
_ = plt.hist(0.1*m1 + 0.9*m2, bins=40)


Can you spot different bells? I certainly can't.