In order to improve performance of my Gaussian Mixture Model based classifier, I was recommended to start with a single multivariate Gaussian, estimate its parameters, and "split" it into two mixtures, reestimate their parameters after several training cycles, and check the classification performance on my cross-validation set, and iterate the process for all the 'split' mixture components.

However I don't really understand what it means to "split" the mixture components and how it is done.

Any help?


1 Answer 1


After fitting the first Gaussian component, you have its mean and covariance. To split it just means to create a new model with two components instead of one. I'd suggest the new components should have the same mean as the first, but with half the covariance added or subtracted, i.e. $\mu_1 = \mu_0 + 0.5 \sigma_0$ and $\mu_2 = \mu_0 - 0.5 \sigma_0$. Intuitively, this shares the data roughly equally between the two new components. Then re-optimise the two-component model and repeat. At each stage, you could take the component with the largest covariance and replace it with two components.

[EDIT] The exact method to initialise the component shouldn't be critical, as the parameters will be optimised further anyway. See also Ueda et al. 2000 for example. One option they suggest is to define the new means by random peturbations of the old component's mean and with unit covariance.

  • $\begingroup$ @ dcorney: I'm not sure I fully understood... how can you subtract the mean (nxd vector) by the covariance matrix (dxd matrix)? Also, what should be the covariance matrices for the split Gaussians? They better not be too linearly dependent... $\endgroup$
    – ladiesman
    Aug 1, 2015 at 3:31
  • $\begingroup$ Sorry for the confusion; my answer was supposed to be more intuitive than an explicit algorithm. The goal is to remove one component and create two similar but different components from it. I don't think the exact method is critical, as the parameters are being re-estimated anyway. $\endgroup$
    – dcorney
    Aug 1, 2015 at 7:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.