# Gaussian Mixture Model $p(x_i | z_i = k)$ a likelihood or probability?

In Gaussian Mixture models, the probability of observing the data $$x$$ given that it was generated from $$M$$ gaussian models is given by the following equation $$p(x) = \sum_{k=1}^m p(x|z=k)p(z = k)$$

People usually refer $$p(x|z=k)$$ as the probability that gaussian $$k$$ generated the data $$x$$ and replaces it with the gaussian density function $$N(\mu_k,\Sigma_k)$$. However, $$N(\mu_k,\sigma_k)$$ is the pdf of the gaussian which represents the likelihood of observing $$x$$ rather than a probability since for continuous distributions the probability for a single data point is 0 although the likelihood is proportional to the probability. Is it considered a probability or a likelihood ?

For a Gaussian mixture, the functions $$p(x)$$ and $$p(x|Z=k)$$ are probability densities, not probability mass functions. In case the mixture model has parameters $$\theta$$ like $$\mu_k$$ and $$\sigma_k$$, the likelihood function is the product of the $$p_\theta(x_i)$$'s $$\ell(\theta) =\prod_{i=1}^n p_\theta(x_i)\tag{1}$$ seen as a function of $$\theta$$ for a given sample $$(x_1,\ldots,x_n)$$, where, e.g., $$p_\theta(x_i) = \sum_{k=1}^m \mathbb P(Z_i = k) p(x_i | \mu_k,\sigma_k)$$ Formally, the case $$n=1$$ makes $$p_\theta(x_1)$$ a likelihood as well, although estimating $$\theta$$ from a single observation is of little interest.
• the y-axis of the probability densities is essentially the likelihood of observing x ? is $p_{\theta}(x_i) = \sum_{k=1}^m p(z_i = k) p(x_i | \mu_k,\sigma_k)$ – calveeen Aug 6 '20 at 6:06
• How would we interpret $p(x)$ in terms of a probability density function ? Since $x$ is a given data point and not a random variable – calveeen Aug 6 '20 at 6:13
• I am not certain we are discussing the same objects. For me, likelihood means the likelihood function $\ell(\theta)$, introduced by R.A. Fisher, that is used in estimation by maximum likelihood. It is defined as (1) in my answer. It is equal to the density (1) of the sample $(X_1,\ldots,X_n)$ at the realised sample $(x_1,\ldots,x_n)$. – Xi'an Aug 6 '20 at 6:56