Consider a univariate probability density function $p(x)$ that is a mixture of $2$ probability density functions with weights $\eta, 1-\eta$ and $\eta\in (0,1)$: $$ p(x)=(1-\eta)g(x)+\eta f(x) \hspace{1cm} \forall x \in \mathbb{R} $$

Suppose we know $p(x)$ and $g(x)$ at every $x\in \mathbb{R}$. Our professor made the following point: without further restrictions we cannot back out $\eta, f(x)$. To see that, it is sufficient to observe that $$ (1-\eta)g(x)+\eta f(x)=(1-\eta/2)g(x)+\eta/2 (-g(x)+2f(x)) $$ Hence, $p(x)$ can be generated by $\{\eta, f(x)\}$ and by $\{\eta/2, -g(x)+2f(x)\}$.

Question: I see the point. However, I don't understand how we are sure that $(g(x)+2f(x))$ is a probability density function. Specifically, it is not necessarily positive. Can't that requirement allow to get rid of the second solution?

  • 1
    $\begingroup$ I think you have a mistake in the second formula. It should be $\left(1 - \frac{\eta}{2}\right) g\left(x\right) + \frac{\eta}{2} \left(2f\left(x\right) - g\left(x\right)\right)$. Then, it integrates to 1. But there is another problem - you are not promised that density is positive, thus it is not a distribution. $\endgroup$ – tmrlvi Feb 5 at 16:17
  • $\begingroup$ Thanks. I've corrected the typo. Am I right to say then that the point of my professor does not actually show what he wanted to say? $\endgroup$ – user3285148 Feb 5 at 16:29
  • $\begingroup$ First, their point stands because they showed that exist case where the pair $\left(f\left(x\right), \eta\right)$ is not uniquely recoverable. Note that in fact, this example can be strengthened to show that you can always find other decompositions. $\endgroup$ – tmrlvi Feb 5 at 16:59
  • $\begingroup$ Thanks. Is this correct: they have shown that $p(x)$ can be generated by $\{\eta, f(x)\}$ with $2f(x)\geq g(x)$ and by $\{\eta/2, -g(x)+2f(x)\}$ $\endgroup$ – user3285148 Feb 5 at 17:20
  • $\begingroup$ Yes, though I would phrase it as "they have shown that, in the case where $2f\left(x\right) \ge g\left(x\right)$, $p\left(x\right)$ can be generated by ..." $\endgroup$ – tmrlvi Feb 5 at 17:23

The following presents a general way to recover another decomposition of a mixture model. In essence, we can always move some of the mass covered by $g\left(x\right)$ to the second component.

Let $\beta>\frac{\eta}{1-\eta}$ and $\alpha=\frac{1}{1+\frac{1}{\beta}}>\eta>0$ (so that $\frac{1}{\alpha}-\frac{1}{\beta}=1$). Note that $$p\left(x\right)=\left(1-\frac{\eta}{\alpha}\right)g\left(x\right)+\frac{\eta}{\alpha}\left(\alpha f\left(x\right)+\alpha\frac{g\left(x\right)}{\beta}\right) $$

Note that $\frac{\eta}{\alpha} \in \left(0, 1\right)$ and that $$ \int_{-\infty}^{\infty}\alpha f\left(x\right)+\alpha\frac{g\left(x\right)}{\beta}dx=\alpha+\alpha\frac{1}{\beta}=\alpha+\alpha\left(\frac{1}{\alpha}-1\right)=1 $$

Thus the pair $\left(\frac{\eta}{\alpha}, \alpha f\left(x\right) + \frac{\alpha}{\beta}g\left(x\right)\right)$ generates $p\left(x\right)$ from $g\left(x\right)$.

We can set $\beta = 2 \frac{\eta}{1 - \eta}$, and we get

$$p\left(x\right) = \left(1-\frac{1+\eta}{2}\right)g\left(x\right)+\frac{1+\eta}{2}\left(\frac{2\eta}{1+\eta}f\left(x\right)+\frac{1-\eta}{1+\eta}g\left(x\right)\right)$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.