Let us follow the convention that a lower information criteria score is considered better.

Suppose we have a ground-truth Gaussian mixture model (GMM) with $k$ components. Suppose also that we (1) have independent samples from an unknown GMM, (2) fit the GMM using EM, and then (3) evaluate the model using an information criterion score (e.g., AIC, BIC, CAIC, etc.). Let $n$ denote the number of components for a particular GMM fit on the samples. So we may fit with $n=1,2,3,\dots$ and obtain the corresponding information criterion scores.

For my particular problem, I only need to determine whether there exists one component or more than one component using the samples.

My question is therefore this: if $k>2$, will the information criterion score of a GMM fit with two components ($n=2$) always be lower than the information criterion score of a GMM fit with one component ($n=1$) in the sample limit (provided that the sample actually does come from a mixture of Gaussians)? In other words, does two components always fit better than one component when the ground truth has more than two components?

Ideally, I would just like to test $n=1$ and $2$ instead of a bunch more, since it would speed up my code substantially.

  • $\begingroup$ How much data do you have? I would think generalization error and/or parameter uncertainty (e.g. via cross validation or a train/test split of the data) might be more suitable than *IC. (This query also gives related questions.) $\endgroup$ – GeoMatt22 Sep 19 '16 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.