I am a biologist trying to understand expectation maximization for a mixture of two Gaussian distributions. I think I understand how to deal with the means of the two distributions, but I don't know how to deal with the standard deviation. I have looked at Wikipedia but the math is a little hard for me to follow.

My understanding of the process is as follows:

You have two groups of individuals. Each group has a different mean and standard deviation. You know the Overall Probability of belonging to each group but you do not know which individual belongs to which group.

You start with your best guess at the mean and standard deviation of both groups. Then, for each individual, you calculate its individual probability of belonging to each group. This happens as follows for a single individual:

Use the Gaussian distributions to calculate the probability of each distribution producing an individual with that value. (i.e. just reading up from the $x$ axis of the graph and getting the $y$ for that value.)

Multiply each probability by the Overall Probability of belonging to that group. (Lets call the resulting values $a$ and $b$.)

For the probability of the individual belonging to group a calculate ${a \over a+b}$. Do the same for the probability of belonging to group b.

When this process has been completed for all individuals, use these probabilities to calculate a weighted mean for both groups - the mean is calculated as normal, except each individual value is multiplied by its chance of belonging to that group.

Once the means are calculated, one can calculate the individual probabilities again, except that this time we update the means of the two distributions with the previously calculated means.

I understand how to calculate the means, but I do not know how to use the probability information to calculate the standard deviation in order to fully update the estimate of the distributions.

  • $\begingroup$ The updated mean is the weighted average of the observations and the updated variance is the weighted sum of squares of the observations. $\endgroup$ – Xi'an May 7 '18 at 16:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.