Can Gaussian Mixture Model Clustering tell me something about the distribution of my data?

Question

I have 10,000 vectors originating from 5 separate classes (2,000 each). I use Gaussian Mixture Model clustering (in Python) to cluster the 10,000 vectors, telling the algorithm to cluster the data into 5 clusters:

gmm = GaussianMixture(n_components=5, covariance_type='full',random_state=0)

Then I find out the predicted cluster allocation of each vector and compare with the original classes:

y_pred = gmm.fit_predict(data)
etc.

I get very high accuracy (>98%), which means the algorithm is successfully able to cluster the data into its original classes.

Now I'm wondering whether this allows me to conclude anything about the distribution of my data. Since the GMM uses a mixture of 5 gaussians to cluster the data, does this mean I can assume:

1. each of my 5 classes is approximately gaussian?
2. the distribution of the 10,000 vectors is well represented by a mixture of 5 gaussians (the means/covariances of the gaussians being the cluster means/covariances that the GMM calculates)?

Why (not)?

I've tried a series of tests for normality on the individual classes (Anderson–Darling, Mardia’s test for multivariate normality, etc.) and these indicate that the individual classes are NOT normally distributed.

Can somebody explain how it's possible for a model that uses a mixture of gaussians to fit my data so well, but that it doesn't seem to imply much about the distribution of my data. Am I missing something obvious?

Answer 1

If your classes are well separated a GMM may fit your data well but it says nothing about the underlying distributions. For example, imagine the data points in each class map to a class specific point (i.e. each class is "distributed" as Dirac delta function). Here, a GMM model could fit these classes effectively by placing the mean of each component on the single point.

For example, we could have a set of data points with a single feature and two classes. If the classes are far apart, they can be well separated by a GMM even though the underlying class distributions are not normal.

Answer 2

Imagine a data set with 5 well separated blocks of uniform data.

GMM will work fine. It will approximate the uniform blocks with Gaussians, but that is good enough to solve this problem.