Mixture of two components

Question

Consider a univariate probability density function $p(x)$ that is a mixture of $2$ probability density functions with weights $\eta, 1-\eta$ and $\eta\in (0,1)$: $$ p(x)=(1-\eta)g(x)+\eta f(x) \hspace{1cm} \forall x \in \mathbb{R} $$

Suppose we know $p(x)$ and $g(x)$ at every $x\in \mathbb{R}$. Our professor made the following point: without further restrictions we cannot back out $\eta, f(x)$. To see that, it is sufficient to observe that $$ (1-\eta)g(x)+\eta f(x)=(1-\eta/2)g(x)+\eta/2 (-g(x)+2f(x)) $$ Hence, $p(x)$ can be generated by $\{\eta, f(x)\}$ and by $\{\eta/2, -g(x)+2f(x)\}$.

Question: I see the point. However, I don't understand how we are sure that $(g(x)+2f(x))$ is a probability density function. Specifically, it is not necessarily positive. Can't that requirement allow to get rid of the second solution?

Answer 1

The following presents a general way to recover another decomposition of a mixture model. In essence, we can always move some of the mass covered by $g\left(x\right)$ to the second component.

Let $\beta>\frac{\eta}{1-\eta}$ and $\alpha=\frac{1}{1+\frac{1}{\beta}}>\eta>0$ (so that $\frac{1}{\alpha}-\frac{1}{\beta}=1$). Note that $$p\left(x\right)=\left(1-\frac{\eta}{\alpha}\right)g\left(x\right)+\frac{\eta}{\alpha}\left(\alpha f\left(x\right)+\alpha\frac{g\left(x\right)}{\beta}\right) $$

Note that $\frac{\eta}{\alpha} \in \left(0, 1\right)$ and that $$ \int_{-\infty}^{\infty}\alpha f\left(x\right)+\alpha\frac{g\left(x\right)}{\beta}dx=\alpha+\alpha\frac{1}{\beta}=\alpha+\alpha\left(\frac{1}{\alpha}-1\right)=1 $$

Thus the pair $\left(\frac{\eta}{\alpha}, \alpha f\left(x\right) + \frac{\alpha}{\beta}g\left(x\right)\right)$ generates $p\left(x\right)$ from $g\left(x\right)$.

We can set $\beta = 2 \frac{\eta}{1 - \eta}$, and we get

$$p\left(x\right) = \left(1-\frac{1+\eta}{2}\right)g\left(x\right)+\frac{1+\eta}{2}\left(\frac{2\eta}{1+\eta}f\left(x\right)+\frac{1-\eta}{1+\eta}g\left(x\right)\right)$$