0
$\begingroup$

I am using Gaussian Mixture Models (GMM) to fit a small data set with ~60 observations and 4 dimensions. This data was generated from the raw data with 14 dimensions after retaining principal components with eigen values >= 1.

I am using the mclust package with a small modification that allows to calculate AICc (following Burnham and Anderson) for each fit. It calculates the best fitting model by searching over a given number of clusters (e.g. G=1:10).

The question is how to set the upper limit on the number of clusters to search for?

Here is the plot for my data when using the model "EEE" in mclust. The AICc drops dramatically at k=11. This is the best fit across all the models provided in mclust. This does not "look" right but it will be better to have a good justification to limit the search space. AICc for different number of clusters

$\endgroup$
1
$\begingroup$

60 instances is much to little.

You probably need 60 instances to get a meaningful location estimate of a single cluster. At 10 clusters, the average cluster size is just 6. Many clusters will likely be based on a single data point! You may even have duplicates? Then at this "flipping" point, every point is exactly encoded in your model. Afterwards, you even get a lot of redundancy, which is probably why the quality decreases again - it is becoming ambiguous, with no more fresh data available at all.

You are badly overfitting your data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.