Applying Akaike Information Criterion on collection of Gaussian fits I am trying to apply Akaike Information Criterion on a collection of Gaussian mixtures fitted on some data points. My question is, can I use AIC even if the number of components of Gaussian mixtures are different?
 A: You $can$ use AIC to compare models, and many do. But it's not necessarily the best way to go, and often chooses too many components. See 
https://www.stat.washington.edu/research/reports/2009/tr559.pdf
http://jaqm.ro/issues/volume-3,issue-4/pdfs/fonseca.pdf
http://www.academia.edu/545182/Mixturemodel_cluster_analysis_using_information_theoretical_criteria
The general consensus appears to be that BIC tends to preform much better.
What is somewhat surprising to me is that I can't seem to find any papers discussing $why$ AIC does poorly, and the answer seems very obvious: the simple model is on the boundary of the more complicated model. This causes a lot of complications, as is well documented in case of likelihood ratio tests. The parallels between a likelihood ratio tests and preforming AIC on nested models is very obvious. So I'm a little surprised this paper hasn't appeared yet. Maybe I just haven't found it. 
A: Yes. This is exactly the kind of situation AIC is designed for; indeed, if you were comparing models with the same number of parameters, maximizing AIC would be reduced to just comparing likelihoods.
