Let us follow the convention that a lower information criteria score is considered better.
Suppose we have a ground-truth Gaussian mixture model (GMM) with $k$ components. Suppose also that we (1) have independent samples from an unknown GMM, (2) fit the GMM using EM, and then (3) evaluate the model using an information criterion score (e.g., AIC, BIC, CAIC, etc.). Let $n$ denote the number of components for a particular GMM fit on the samples. So we may fit with $n=1,2,3,\dots$ and obtain the corresponding information criterion scores.
For my particular problem, I only need to determine whether there exists one component or more than one component using the samples.
My question is therefore this: if $k>2$, will the information criterion score of a GMM fit with two components ($n=2$) always be lower than the information criterion score of a GMM fit with one component ($n=1$) in the sample limit (provided that the sample actually does come from a mixture of Gaussians)? In other words, does two components always fit better than one component when the ground truth has more than two components?
Ideally, I would just like to test $n=1$ and $2$ instead of a bunch more, since it would speed up my code substantially.