Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite
3

I have a data set that behaves approx. Standard normal. It is an image where each observation is a pixel intensity. I want to cluster this into three different sets by fitting a 3-gaussian mixture model. I want one cluster to be "significantly" smaller than the mean (0) one to be around the mean and one "significantly" larger than the mean.

I find the initial moments and mixture coefficients by determining intial sets and calculatins the corresponding sample means and std's. This is:

  • Set1:=(x|x smaller than -2std)
  • Set2:=(x|x in (-0.5std,0.5std))
  • Set3:=(x|x larger than 2std)

Taking the sample means and std's of these sets I can estimate initial moments and mixture coefficients to later use the EM algorithm to get 'optimum' parameters for the 3-gaussian mixture.

Using the pymix package I do: 

from pymix import mixture

# standard deviation of whole data set
standev= np.std(x,ddof=1)

# Initial Sets for Negative-Change, No-Change and Positve Change    
CN = x[x<(-2*standev)]
NC = x[ (x > (-1*standev)/2) & (x < (standev/2))] 
CP = x[x>(2*standev)]


# Initial Means and Variances   
CNimean = np.mean(CN)
CNistd = np.std(CN)

NCimean = np.mean(NC)
NCistd = np.std(NC)

CPimean = np.mean(CP)
CPistd = np.std(CP)

# Load data for mixture model
data = mixture.DataSet()

data.fromArray(np.squeeze(np.asarray(x)))

# Initialize mixture model

n1 = mixture.NormalDistribution(CNimean,CNistd)
n2 = mixture.NormalDistribution(NCimean,NCistd)
n3 = mixture.NormalDistribution(CPimean,CPistd)

m = mixture.MixtureModel(3,[0.01,0.98,0.01], [n1,n2,n3])

# Resolver el Mixture model

c = m.EM(data,50,0.1)

What I'm no sure on how to estimate is the proportion. For pymix's em to solve this mixture model you also need to supply an initial guessing of the proportion of observations that belong to each of the three gaussians (sum = 1). And after trying some judicious guessing, for example 1% 98% 1% as the extremal sets are before -2stds and after 2stds I end up with a mixture having for example 0.03 0.02 and 0.95 in that order, and with parameters that dont make a lot of sense.

Does anyone know what I might be doing wrong? or is my approach wrong all over? Any help would be deeply appreciatd!

Greetings!

share|improve this question
Try your initial guesses for the proportions to be a little more equal, say, 25% / 50% / 25%. Don't forget the data isn't normal (if it were, you'd not be needing to fit a mixture to it) so the 1% - 2stds rule won't apply to it. – jbowman Jun 4 '12 at 18:37
Could you describe exactly how the parameter "don't make a lot of sense"? How have you compared those parameters to the data? – whuber♦ Jun 4 '12 at 19:12
I trI tried using 25% / 50% / 25% and I get 9.1% / 14 % / 76% in that order with parameters mu1=-0.29, sigma1=0.99 mu2=0.55,sigma2=0.86 mu3=0.005,sigma3=0.52. Which is not bad but it's not entirely correct. I would have to show you the image of the 3 distributions and that of the data, for example a histogram. The algorithm is also very sensible to the number of iterations/ tolerance I choose and to this initial guessing of the proportions. Any idea on how to guess these proportions more judiciously? Also the final guessings tend to get worse while the size of the image increases @whuber – JEquihua Jun 4 '12 at 20:07
If you can make the image available on the Web somewhere, then please do so and edit your question to link to it. The relevant histograms are (i) the data and (ii) the mixture model (as a whole, not component by component). (A probability plot would be even better.) It is true that the algorithm is sensitive to initial values. But what do you mean by "entirely correct"? Even if you test with simulated data (from a known mixture), chance will cause the fitted parameters to differ from the intended ones. – whuber♦ Jun 4 '12 at 20:25
Heres is the link, if maybe not very elegant. dropbox.com/sh/b9alsky18xxb8cs/WrwQR1fdaY Also, as it can be seen in the histogram the data is quite close to being normal, I just want to find 3-normals that can best fit it. Im following an article that i also uploaded to that link, section 3. Is there something I am missing? Thank you for taking this time to try and help me, it's grand. @Whuber – JEquihua Jun 4 '12 at 22:55

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.