1
$\begingroup$

In Gaussian Mixture models, the probability of observing the data $x$ given that it was generated from $M$ gaussian models is given by the following equation $$p(x) = \sum_{k=1}^m p(x|z=k)p(z = k)$$

People usually refer $p(x|z=k)$ as the probability that gaussian $k$ generated the data $x$ and replaces it with the gaussian density function $N(\mu_k,\Sigma_k)$. However, $N(\mu_k,\sigma_k)$ is the pdf of the gaussian which represents the likelihood of observing $x$ rather than a probability since for continuous distributions the probability for a single data point is 0 although the likelihood is proportional to the probability. Is it considered a probability or a likelihood ?

$\endgroup$
1
$\begingroup$

For a Gaussian mixture, the functions $p(x)$ and $p(x|Z=k)$ are probability densities, not probability mass functions. In case the mixture model has parameters $\theta$ like $\mu_k$ and $\sigma_k$, the likelihood function is the product of the $ p_\theta(x_i)$'s $$\ell(\theta) =\prod_{i=1}^n p_\theta(x_i)\tag{1}$$ seen as a function of $\theta$ for a given sample $(x_1,\ldots,x_n)$, where, e.g., $$p_\theta(x_i) = \sum_{k=1}^m \mathbb P(Z_i = k) p(x_i | \mu_k,\sigma_k)$$ Formally, the case $n=1$ makes $p_\theta(x_1)$ a likelihood as well, although estimating $\theta$ from a single observation is of little interest.

$\endgroup$
  • $\begingroup$ the y-axis of the probability densities is essentially the likelihood of observing x ? is $p_{\theta}(x_i) = \sum_{k=1}^m p(z_i = k) p(x_i | \mu_k,\sigma_k)$ $\endgroup$ – calveeen Aug 6 '20 at 6:06
  • $\begingroup$ How would we interpret $p(x)$ in terms of a probability density function ? Since $x$ is a given data point and not a random variable $\endgroup$ – calveeen Aug 6 '20 at 6:13
  • $\begingroup$ I am not certain we are discussing the same objects. For me, likelihood means the likelihood function $\ell(\theta)$, introduced by R.A. Fisher, that is used in estimation by maximum likelihood. It is defined as (1) in my answer. It is equal to the density (1) of the sample $(X_1,\ldots,X_n)$ at the realised sample $(x_1,\ldots,x_n)$. $\endgroup$ – Xi'an Aug 6 '20 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.