1
$\begingroup$

I have a latent Gaussian model with unknown parameters $\mu$ and $\sigma^2$. I can estimate these parameters using MLE and an EM-ish algorithm. However the solution is not stable; I end up in local maxima depending on the order with which I withhold data during EM. (Though the solutions are close to each other.)

So, i can run the algorithm 50 times and get 50 different estimates for $\mu$ and $\sigma^2$. I think the best estimate for the true value of $\mu$ is the average over all of these estimates. However I am unsure whether the best value for $\sigma^2$ is simply the average of all estimates for $\sigma^2$. Is that correct?

Edit: Now that I'm thinking about it, perhaps averaging the parameters is a bad idea, because it doesn't actually represent a maximum, and I should just choose the $(\mu, \sigma^2)$ that is the max of the max.

$\endgroup$
  • 1
    $\begingroup$ It might be worth trying the median parameter values, rather than the average parameter values. $\endgroup$ – James Phillips Mar 19 at 23:41
0
$\begingroup$

This slightly depends on your goals. If you just want maximum-likelihood estimate than taking the "max of the max" is definitely sensible. If you want something that would, on average, be close to a result you would get when repeating the procedure then a mean or median might make more sense.

Also, if there actually are multiple local maxima of the likelihood, this might indicate serious issues with your model (which I can't judge since I don't see the full model). In particular, if the likelihood is multimodal, MLE might not be very representative of the set of plausible parameter values. On the other hand, from your description it looks more like the differences are an artifact of the way your algorithm works - if so, then running the alg multiple times and choosing the max is probably OK. Even better, this might motivate some improvements to the algorithm to make the results more stable (I would wildly guess that a tweak to termination criterion might be able to resolve this).

$\endgroup$
0
$\begingroup$

Ideally I would think of the following approach

  • Mean : Take the average of the mean
  • StdDev : Take the standard deviation that is the max

The reason for taking the maximum standard deviation is because we want to take the most conservative approach

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.