Understanding the log-likelihood calculation of sklearn Gaussian mixture model

Question

I am trying to understand how the Scipy is calculating the score of a sample in the Gaussian Mixture model(log-likelihood).

Below is the equation I got for log-likelihood from the book C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.

In my code I am using the following parameters:

gmm = GaussianMixture(n_components=2, covariances_type = 'diag',random_state=0)

I can run gmm.score(X) to get the log-likelihood of the sample. When I investigated the source code, it was not using the determinant or inverse of the covariance. Instead, it was using Cholesky precision matrix.

def _estimate_log_prob(self, X):
   return _estimate_log_gaussian_prob(
   X, self.means_, self.precisions_cholesky_, self.covariance_type)



def _estimate_log_gaussian_prob(X, means, precisions_chol, covariance_type):
    log_det = _compute_log_det_cholesky(
    precisions_chol, covariance_type, n_features)

[...]

    elif covariance_type == 'diag':
       precisions = precisions_chol ** 2
       log_prob = (np.sum((means ** 2 * precisions), 1) -
                  2. * np.dot(X, (means * precisions).T) +
                np.dot(X ** 2, precisions.T))

[...]
    return -.5 * (n_features * np.log(2 * np.pi) + log_prob) + log_det


def _compute_log_det_cholesky(matrix_chol, covariance_type, n_features):
[...]
    elif covariance_type == 'diag':
        log_det_chol = (np.sum(np.log(matrix_chol), axis=1))
[...]
    return log_det_chol

This post explained the mathematics behind it, which is great. But I am confused about the following:

1. If = np.sum(np.log(matrix_chol), would = np.prod(matrix_chol)?
2. How is = (np.sum((means ** 2 * precisions_chol ** 2), 1) - 2. * np.dot(X, (means * precisions_chol ** 2).T) + np.dot(X ** 2, precisions_chol ** 2.T))

I would appreciate any answers or feedback from anyone.

Have a great day!

Answer 1

1. The determinant of a diagonal matrix is the product of the diagonal elements. You can prove this using cofactor expansion along the first column and applying the induction hypothesis.

2. The second equation just applies the distributive property of multiplication.