I am using Gaussian Mixture Models (GMM) to fit a small data set with ~60 observations and 4 dimensions. This data was generated from the raw data with 14 dimensions after retaining principal components with eigen values >= 1.
I am using the mclust package with a small modification that allows to calculate AICc (following Burnham and Anderson) for each fit. It calculates the best fitting model by searching over a given number of clusters (e.g. G=1:10).
The question is how to set the upper limit on the number of clusters to search for?
Here is the plot for my data when using the model "EEE" in mclust. The AICc drops dramatically at k=11. This is the best fit across all the models provided in mclust. This does not "look" right but it will be better to have a good justification to limit the search space.