I have a problem where I am trying to classify data into two groups using a single parameter. The distribution of this parameter is Gaussian for two groups, so what I'm dealing with is two overlapping Gaussians. If I knew what the parameters of both Gaussians, I could just use Bayes theorem to calculate the posterior probability for each data point directly.
However, I only know the means and standard deviations, so I use what I think is a very simple Empirical Bayes method to solve for the relative magnitude of each Gaussian. It's an iterative approach, I start with a guess for the relative magnitude of each Gaussian, calculate the posterior probabilities and then sum them for each group to get an estimate of the relative magnitude of each Gaussian which I then plug back in to get a new posterior probability. I continue this until the difference in the ratio of the sums of posterior probabilities between iterations is less than 1%.
So this works well, but I don't understand why? Like why does the sum of posterior probabilities always push the solution in the right direction, and why does it converge?
I understand I could just fit two Gaussian to the data, but this approach works better than direct fitting, and I don't understand why it does. I appreciate any help, thanks.