I am learning mixture model. I understand what does it means, but I still do not understand some contexts. I really do not understand the following:

"The moments of a mixture random variables are linear combinations of the corresponding moments of the individual random variables." quote from" this site

Any help please?


Let $f_1$ and $f_2$ be two distributions and let $0<p<1$. You generate $X$, an observation from the mixture of $f_1$ and $f_2$, as follows: First generate $U\sim Unif(0,1)$. If $U\leq p$ then generate $Y\sim f_1$ and set $X=Y$. Otherwise, generate $Y\sim f_2$ and set $X=Y$.

It is therefore clear that for any set $A$, $$P(X\in A)$$

$$=P(X\in A|U\leq p)P(U\leq p)+P(X\in A|U>p)P(U>p)$$

$$=p\int_A f_1(x)dx+(1-p)\int_A f_2(X)dx$$

This is because given $U\leq p$, $X$ is coming from $f_1$.

Thus the density of $X$ will simply be $f_X(x)=pf_1(x)+(1-p)f_2(x)$.

So he $r$-th moment of $X$ will be $E(X^r)=\int_\mathbb R x^rf_X(x)dx=p\int_\mathbb Rx^rf_1(x)dx+(1-p)\int_\mathbb R x^rf_2(x)dx =pm_1(r)+(1-p)m_2(r)$

where $m_i(r)$ is the $r$-th moment under $f_i$.

| cite | improve this answer | |
  • $\begingroup$ Thank you so much for your help. Then, what does "linear" mean here. Also, do we have $1-p$ because we given $U >p$? $\endgroup$ – Silver_80 Oct 5 '17 at 6:31
  • 1
    $\begingroup$ Linear means the convex combination of the individual moments. In general if the mixture distribution has probabilities $p_1,p_2,...,p_k$ and the $r$-th moment for $f_i$ ($1\leq i\leq k$) is $m_i(r)$ then the r-th moment of mixture is $\sum_{i=1}^k p_im_i(r)$. This is a linear combination of the individual moments. And yes you have $1-p$ due to $P(U>p)$. $\endgroup$ – Landon Carter Oct 5 '17 at 6:42
  • $\begingroup$ Amazing answer and comment. Thank you so much for your help. $\endgroup$ – Silver_80 Oct 5 '17 at 6:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.