Timeline for Convergence of EM for Mixture of Gaussians
Current License: CC BY-SA 3.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 14, 2018 at 0:28 | comment | added | Cliff AB | @Dorain: actually, this issue has nothing to do with the value of $p_1$ or $p_2$, as long as they are both positive. I merely fix them equal to 0.5 for simplicity, as if the likelihood is unbounded for fixed values of $p_1, p_2$ then it is unbounded if they are allowed to vary as well. | |
| Feb 14, 2018 at 0:14 | comment | added | Dorian | The problem here is with your assumption that $p_1=p_2 = 0.5$. This would never happen with $\mu_2=x_1$, since when you do the Bayesian update, this would ultimately imply that $p_2$ is very small compared to $p_1$ (since you're implying that the second Gaussian is completely determined by one point). Thus, this scenario won't ever happen (at least the way you've described it). | |
| May 22, 2015 at 19:49 | vote | accept | user27886 | ||
| May 21, 2015 at 6:05 | comment | added | Xi'an | While it is exact the maximum of the likelihood is equal to $+\infty$, I have never observed an EM algorithm converging to that degenerate solution. | |
| May 20, 2015 at 23:54 | history | edited | Cliff AB | CC BY-SA 3.0 |
had written "bounded" when I meant "unbounded" in final paragraph
|
| May 20, 2015 at 23:19 | comment | added | Cliff AB | And yes, the issue is that one of the components can collapse around a single point. | |
| May 20, 2015 at 23:10 | comment | added | Cliff AB | Sorry, just cleaned it up a little. Does that make more sense? | |
| May 20, 2015 at 23:07 | history | edited | Cliff AB | CC BY-SA 3.0 |
explained unbounded nature of likelihood in a more thorough manner
|
| May 20, 2015 at 23:02 | comment | added | user27886 | I am bit confused at what $x_1$, $\hat s$, and $\bar x$ are referring to as well. Are you suggesting that the second gaussian is collapsing on one point, approaching a delta function? | |
| May 20, 2015 at 22:05 | comment | added | whuber♦ | I was not able to follow this argument, because the nature of the likelihood function must depend on the data but you make no reference to them at all. I suspect you might also need to make clearer distinctions between the true parameters and potential estimates of them. The wildly varied and inconsistent notation (bars, hats, subscripts, no subscripts, greek letters, and latin letters) provides no apparent help in deciphering what you really mean. Do you think you could clarify this answer? | |
| May 20, 2015 at 21:21 | comment | added | Cliff AB | Just to clarify a little: the reason the likelihood approaches $\infty$ when $\epsilon$ approaches 0 is that the likelihood contribution of $x_1$ approaches $\infty$ as component 2 "closes in" on it. The likelihood contributions of the other observations are prevented from approaching $-\infty$ by component 1. | |
| May 20, 2015 at 20:59 | history | answered | Cliff AB | CC BY-SA 3.0 |