# Mixture model and Pymix (python package for mixture models)

I have a data set that behaves approx. Standard normal. It is an image where each observation is a pixel intensity. I want to cluster this into three different sets by fitting a 3-gaussian mixture model. I want one cluster to be "significantly" smaller than the mean (0) one to be around the mean and one "significantly" larger than the mean.

I find the initial moments and mixture coefficients by determining intial sets and calculatins the corresponding sample means and std's. This is:

• Set1:=(x|x smaller than -2std)
• Set2:=(x|x in (-0.5std,0.5std))
• Set3:=(x|x larger than 2std)

Taking the sample means and std's of these sets I can estimate initial moments and mixture coefficients to later use the EM algorithm to get 'optimum' parameters for the 3-gaussian mixture.

Using the pymix package I do:

from pymix import mixture

# standard deviation of whole data set
standev= np.std(x,ddof=1)

# Initial Sets for Negative-Change, No-Change and Positve Change
CN = x[x<(-2*standev)]
NC = x[ (x > (-1*standev)/2) & (x < (standev/2))]
CP = x[x>(2*standev)]

# Initial Means and Variances
CNimean = np.mean(CN)
CNistd = np.std(CN)

NCimean = np.mean(NC)
NCistd = np.std(NC)

CPimean = np.mean(CP)
CPistd = np.std(CP)

# Load data for mixture model
data = mixture.DataSet()

data.fromArray(np.squeeze(np.asarray(x)))

# Initialize mixture model

n1 = mixture.NormalDistribution(CNimean,CNistd)
n2 = mixture.NormalDistribution(NCimean,NCistd)
n3 = mixture.NormalDistribution(CPimean,CPistd)

m = mixture.MixtureModel(3,[0.01,0.98,0.01], [n1,n2,n3])

# Resolver el Mixture model

c = m.EM(data,50,0.1)


What I'm no sure on how to estimate is the proportion. For pymix's em to solve this mixture model you also need to supply an initial guessing of the proportion of observations that belong to each of the three gaussians (sum = 1). And after trying some judicious guessing, for example 1% 98% 1% as the extremal sets are before -2stds and after 2stds I end up with a mixture having for example 0.03 0.02 and 0.95 in that order, and with parameters that dont make a lot of sense.

Does anyone know what I might be doing wrong? or is my approach wrong all over? Any help would be deeply appreciatd!

Greetings!

-
Try your initial guesses for the proportions to be a little more equal, say, 25% / 50% / 25%. Don't forget the data isn't normal (if it were, you'd not be needing to fit a mixture to it) so the 1% - 2stds rule won't apply to it. –  jbowman Jun 4 '12 at 18:37
Could you describe exactly how the parameter "don't make a lot of sense"? How have you compared those parameters to the data? –  whuber Jun 4 '12 at 19:12
I trI tried using 25% / 50% / 25% and I get 9.1% / 14 % / 76% in that order with parameters mu1=-0.29, sigma1=0.99 mu2=0.55,sigma2=0.86 mu3=0.005,sigma3=0.52. Which is not bad but it's not entirely correct. I would have to show you the image of the 3 distributions and that of the data, for example a histogram. The algorithm is also very sensible to the number of iterations/ tolerance I choose and to this initial guessing of the proportions. Any idea on how to guess these proportions more judiciously? Also the final guessings tend to get worse while the size of the image increases @whuber –  JEquihua Jun 4 '12 at 20:07
If you can make the image available on the Web somewhere, then please do so and edit your question to link to it. The relevant histograms are (i) the data and (ii) the mixture model (as a whole, not component by component). (A probability plot would be even better.) It is true that the algorithm is sensitive to initial values. But what do you mean by "entirely correct"? Even if you test with simulated data (from a known mixture), chance will cause the fitted parameters to differ from the intended ones. –  whuber Jun 4 '12 at 20:25
Heres is the link, if maybe not very elegant. dropbox.com/sh/b9alsky18xxb8cs/WrwQR1fdaY Also, as it can be seen in the histogram the data is quite close to being normal, I just want to find 3-normals that can best fit it. Im following an article that i also uploaded to that link, section 3. Is there something I am missing? Thank you for taking this time to try and help me, it's grand. @Whuber –  JEquihua Jun 4 '12 at 22:55