I have a latent Gaussian model with unknown parameters $\mu$ and $\sigma^2$. I can estimate these parameters using MLE and an EM-ish algorithm. However the solution is not stable; I end up in local maxima depending on the order with which I withhold data during EM. (Though the solutions are close to each other.)
So, i can run the algorithm 50 times and get 50 different estimates for $\mu$ and $\sigma^2$. I think the best estimate for the true value of $\mu$ is the average over all of these estimates. However I am unsure whether the best value for $\sigma^2$ is simply the average of all estimates for $\sigma^2$. Is that correct?
Edit: Now that I'm thinking about it, perhaps averaging the parameters is a bad idea, because it doesn't actually represent a maximum, and I should just choose the $(\mu, \sigma^2)$ that is the max of the max.