There seems to be (at least) two ways that one could evaluate the fit of a Gaussian Mixture Model (GMM) to a data set.

First, a probabilistic score, is the log likelihood of a set of points $D$ under the GMM:

$$ \log p(D|\theta) = \sum_{i=1}^{N_{points}} \log f(x_i) $$

where I've defined the GMM probability $f(x_i)=\sum_{k=1}^{N_{components}} \pi_k \mathcal{N}(x_i|\mu_k,\Sigma_k)$. Consider the case that the data are binned, in which we can use the bin center approximation: $$ \log p(D|\theta) = \sum_{i=1}^{N_{bins}} |b_i|\log (f(b_i)\Delta b_i) $$

where $b_i$ is the position of the bin center, |$b_i$| is the number of observations, and $\Delta b_i$ is the bin width.

Second, a density score. Since the binned data is also a probability density we can use density metrics like the $L^2$ norm: $$ \begin{align} L^2(f,D) &= \int_x \left(f(x) - D(x)\right)^2dx \\&= \int_x f^2(x)dx -2\int_x D(x)f(x)dx + \int_xD^2(x)dx \end{align} $$ The first term is easy to compute because it's the sum of integrals over all products of components of $f$, so that's analytic. The last term is just the sum over the squared values of the histogram. The middle term, again using the bin center approximation, is nearly identical to the log-likelihood: $$ \int_x D(x)f(x) = \sum_{i=1}^{N_{bins}}|b_i| f(b_i)\Delta b_i $$ Here are my questions: what are the differences between these "scores"? When should you each of them, or alternatively other scores like the correlation? The appeal (to me) of something like the $L^2$ is that one could compare, using the same metric, multiple GMMs to each other or to $D$ using the same function. On the other hand, the probabilistic score appears to be weighted by the number of observations.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.