Gaussian Mixture Model with labels in Python - Cross Validated most recent 30 from stats.stackexchange.com 2019-07-18T19:54:43Z https://stats.stackexchange.com/feeds/question/369220 http://www.creativecommons.org/licenses/by-sa/3.0/rdf https://stats.stackexchange.com/q/369220 1 Gaussian Mixture Model with labels in Python jbuchel https://stats.stackexchange.com/users/196645 2018-09-28T18:06:28Z 2018-09-28T18:06:28Z <p>I have data <strong>X</strong> and corresponding labels <strong>y</strong> and want to fit a Gaussian Mixture model to it. In Matlab, one has the option of specifying initial labels. I am trying to do the same in Python. This is what I have so far:</p> <pre><code>def mixture(dataset): print("Fitting mixture of gaussians...") from sklearn.mixture import GaussianMixture X = dataset.train y = np.argmax(dataset.train,axis=1) X_test = dataset.test y_test = np.argmax(dataset.test, axis=1) #Compute the means of the individual classes n_classes = len(np.unique(y)) means = np.zeros((n_classes,204)) weights = np.ones(n_classes) * (X.shape/float(n_classes))/X.shape inv_cov = np.zeros((n_classes,204,204)) for j in range(n_classes): means[j,] = np.mean(X[y == j]) inv_cov[j,:,:] = np.linalg.inv(np.cov(X[y == j])) GMModel = GaussianMixture(n_components=16, covariance_type='full', tol=0.001, reg_covar=1e-06, max_iter=100, n_init=1, init_params='kmeans', weights_init=weights, means_init=means, precisions_init=inv_cov, random_state=None, warm_start=False, verbose=0, verbose_interval=10) GMModel.fit(X) #print("Converged: %s" % GMModel.converged_) y_hat = GMModel.predict(X_test) acc = np.sum(y_hat == y_test)/float(len(y_hat)) print("Accuracy of GMM is %s " % acc) </code></pre> <p>1) Just using the precomputed means gives 0.05% accuracy.</p> <p>2) Also using equal weights (X is completely balanced) I am getting 10%.</p> <p>3) I tried initializing the inverse of the covariance matrix, but some covariance matrices are singular so this fails.</p> <p>4) Basically, k-means++ is redundant, since we already know the true cluster centers (mean of the individual points), but it seems that sklearn is still using it (maybe for the inverse covariance matrices as it says in the doc).</p> <p>Is this, in general, the only way to do this? And why am I getting such bad results? In Matlab, I am achieving 98.8% accuracy. Has anyone tried this?</p>