I have data X and corresponding labels y and want to fit a Gaussian Mixture model to it. In Matlab, one has the option of specifying initial labels. I am trying to do the same in Python. This is what I have so far:

def mixture(dataset):
  print("Fitting mixture of gaussians...")
  from sklearn.mixture import GaussianMixture
  X = dataset.train[0]
  y = np.argmax(dataset.train[1],axis=1)
  X_test = dataset.test[0]
  y_test = np.argmax(dataset.test[1], axis=1)
  #Compute the means of the individual classes
  n_classes = len(np.unique(y))
  means = np.zeros((n_classes,204))
  weights = np.ones(n_classes) * (X.shape[0]/float(n_classes))/X.shape[0]
  inv_cov = np.zeros((n_classes,204,204))
  for j in range(n_classes):
    means[j,] = np.mean(X[y == j])
    inv_cov[j,:,:] = np.linalg.inv(np.cov(X[y == j]))

  GMModel = GaussianMixture(n_components=16, covariance_type='full', tol=0.001, reg_covar=1e-06,
                            max_iter=100, n_init=1, init_params='kmeans', weights_init=weights, means_init=means, precisions_init=inv_cov,
                            random_state=None, warm_start=False, verbose=0, verbose_interval=10)

  #print("Converged: %s" % GMModel.converged_)
  y_hat = GMModel.predict(X_test)
  acc = np.sum(y_hat == y_test)/float(len(y_hat))
  print("Accuracy of GMM is %s " % acc)

1) Just using the precomputed means gives 0.05% accuracy.

2) Also using equal weights (X is completely balanced) I am getting 10%.

3) I tried initializing the inverse of the covariance matrix, but some covariance matrices are singular so this fails.

4) Basically, k-means++ is redundant, since we already know the true cluster centers (mean of the individual points), but it seems that sklearn is still using it (maybe for the inverse covariance matrices as it says in the doc).

Is this, in general, the only way to do this? And why am I getting such bad results? In Matlab, I am achieving 98.8% accuracy. Has anyone tried this?

  • $\begingroup$ GMM is a clustering algorithm, hence the cluster allocation values may not be the same as the class label values. In other words, samples with y_test = 1, may be assigned to cluster 2, whose corresponding values could be y_hat = 2. In such a case evaluating the accuracy, using acc = np.sum(y_hat == y_test)/float(len(y_hat)), may not be the best way to evaluate clustering performance. $\endgroup$ – kedarps Oct 2 '18 at 16:21
  • $\begingroup$ Ok I understand. This seems a little stupid to be honest. It was a 12 liner in Matlab and the cluster indices stay the same for the labels. $\endgroup$ – jbuchel Oct 2 '18 at 16:35
  • $\begingroup$ You can't guarantee that the cluster indices will be the same as class labels because of the unsupervised nature of the algorithm. There are several ways to evaluate clustering performance if class labels are available, see here. $\endgroup$ – kedarps Oct 2 '18 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.