# Can someone verify if the following Bayesian Information Criterion (BIC) model selection algorithm is correct for Gaussian mixture models?

I am trying to find an automated way of picking the number of clusters $$K \in \mathbb{N}$$ for unsupervised learning scenarios, specifically for GMM.

I was suggested to use something called the "Bayesian Information Criterion" (BIC) for GMM

The recommended BIC algorithm states,

1. Let $$k = 1$$, (start with one cluster)
2. Run EM algorithm for GMM and obtain mean, variance and mixing coefficients that maximizes the log-likehood, which we refer to as {mean, variance, mixing coefficients}
3. Plug these values into the "magical" formula

$$BIC(k, N) = -2 \times \text{log-likelihood function} (\text{mean, variance, mixing coefficients}) + \log(N) \times k (2d+1)$$ where $$N$$ is the number of data points, $$d$$ is the dimension of the data

1. $$k \leftarrow k + 1$$ and repeat (step 1 - 4) until $$k$$ is very large

2. Pick the $$k$$ that gives the minimum BIC value.

I am not sure if this is correct and would like a second pair of eye to verify. In particular, I am not seeing anything that indicates how I should pick my initial values. I am also not sure how long I should run this algorithm for.

Note: a similar formulation for the BIC is given in this question The bayesian information criterion (BIC) Under the Gaussian model

• The last term in the BIC formula is the number of model parameters, which doesn't look right to me in your case - however this depends on your choice of covariance for the mixture model. I laid that out for the case of a full covariance matrix in my answer here. Generally the procedure you describe seems valid though. Also note that you could handle the choice of k by fitting the model via variational inference. – deemel Dec 31 '19 at 12:41