# What is “mixture” in a gaussian mixture model

We often study Gaussian Mixture model as a useful model in machine learning and its applications.

What is the physical significance of this "Mixture"?

Is it used because a Gaussian Mixture Model models the probability of a number of random variables each with its own value of mean? If not, then what is the correct interpretation of this word.

A distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A is the special case where the components are Gaussians.

For instance, here is a mixture of 25% $N(-2,1)$ and 75% $N(2,1)$, which you could call "one part $N(-2,1)$ and three parts $N(2,1)$":

xx <- seq(-5,5,by=.01)
plot(xx,0.25*dnorm(xx,-2,1)+0.75*dnorm(xx,2,1),type="l",xlab="",ylab="")


Essentially, it's like a recipe. Play around a little with the weights, the means and the variances to see what happens, or look at the two tags on CV.

Yes a Gaussian Mixture is called this way because it is assumed that the observed data come from a Gaussian mixture distribution which consists of $K$ Gaussians with their own means and variances. However, the $K$ classes are latent and so is the indicator telling you which class an observations belongs to.

The goal of mixture modeling is now to estimate the most likely class for each observation. Therefore, Gaussian mixture modeling can be viewed as a missing data problem. Estimation is usually done using the EM algorithm.

• can I assume that the K gaussians are related to one another (through weights such that their sum equals to one) – Upendra Pratap Singh Sep 22 '16 at 10:35
• @Snowbell Yes, usually it is assumed that the weights are normalized in the sense that they sum to 1. For general information about mixtures you might want to check en.wikipedia.org/wiki/Mixture_distribution – tomka Sep 22 '16 at 11:41