# A Gaussian Mixture Model Is a Universal Approximator of Densities

When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following:

A Gaussian mixture model is a universal approximator of densities, in the sense that any smooth density can be approximated with any specific nonzero amount of error by a Gaussian mixture model with enough components.

I only have a basic background in probability and statistics, and I have absolutely no idea what this section is saying.

I would greatly appreciate it if people could please take the time to explain this in a way that is more understandable to someone of my level.

• I applied this idea once, very early in my career, to carry out quantum-mechanical calculations by using a basis of Gaussians to model bonds in organic molecules. After weeks of running into problems I realized, to my chagrin, that the densities I was trying to approximate were so sharply peaked that it was hopeless to develop a good approximation without using a ridiculously large number of Gaussians, which are too smooth for the job. Thus, one should always be cognizant of the distinction between a purely mathematical theorem and the practical needs of computation. – whuber Sep 3 '18 at 13:33

## 1 Answer

The idea is that an arbitrary density on $\mathbb{R}$, $f(\cdot)$, can be approximated by a Gaussian mixture model $$g_k(\cdot;\boldsymbol{\omega,\mu,\sigma})=\sum_{i=1}^k \omega_i \varphi(\cdot;\mu_i,\sigma_i)$$ in the sense that $$\mathfrak{D}(f(\cdot),g_k(\cdot;\boldsymbol{\omega,\mu,\sigma})\stackrel{k\to\infty}{\longrightarrow}0$$ for some specific functional distance $\mathfrak{D}(\cdot,\cdot)$. This result only applies for weaker types of distance.