matlab gmdistribution.fit 'Regularize' - what regularization method?

Question

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance matrix inversible, is convergence of fit algorithm guaranteed? Is some more sophisticated method used here, like shrinkage covariance regularization?

Doc for this function is here.

In general, what do you think of adding small value (for instance 10E-4!) to diagonal elements? Is convergence guaranteed?

Thanks

Answer 1

Convergence depends on the data. If you have well behaved data then you need no regularization. If you have ugly data (like the real world sometimes gives) then you might need regularization.

Yes, it adds to the diagonal of the covariance matrix in order to "stabilize" inversion.

I think of it like ridge regression, it adds a small bias in exchange for smaller bounds on other estimation errors and for better stability. It can also improve "convergence" rates.