5
$\begingroup$

Consider probability density functions $f_{1}\left(x\right)$ and $f_{2}\left(x\right)$ and the mixture distribution $$f_{3}\left(x\right)\equiv pf_{1}\left(x\right)+\left(1-p\right)f_{2}\left(x\right)$$ Assume that the moments associated with $f_{1}\left(x\right)$ are all clearly defined, but the moments of $f_{2}\left(x\right)$ may not necessarily be (e.g. a Cauchy or Levy distribution).

To calculate the expected value one would find $$\int_{-\infty}^{\infty}xf_{3}\left(x\right)dx=p\int_{-\infty}^{\infty}xf_{1}\left(x\right)dx+\left(1-p\right)\int_{-\infty}^{\infty}xf_{2}\left(x\right)dx$$ Presumably if $\int_{-\infty}^{\infty}xf_{2}\left(x\right)dx$ is infinite or undefined, then the expected value only exists when $p=1$.

Is this logic correct, even if $p$ is close to but less than $1$? Does it extend to higher moments as well?

|cite|improve this question
$\endgroup$
4
$\begingroup$

Yes, you are correct. If $X_1 \sim f_1$, $X_2 \sim f_2$, and $X_3 \sim f_3$, then your equation shows that $$E(X_3) = p \cdot E(X_1) + (1-p) \cdot E(X_2)$$ Therefore if either one of $E(X_1)$ or $E(X_2)$ in non-finite/non-existent then $E(X_3)$ will be non-finite/non-existent also if $p \in (0,1)$ - this is true even if $p$ is very near $0$ or $1$.

To get some intuition for this, note that the mixture distribution can be thought of a drawing from $f_1$ with probability $p$ and $f_2$ with probability $1-p$. Bearing that in mind, take an example where $f_1$ is the density of the reciprocal of a standard normal (a distribution with infinite mean), $f_2$ is the standard normal density and $p$ is some very small value (say $.01$). Consider sampling variables that have density $f_3$ - of the $1\%$ that are drawn from $f_1$, there will be some extreme values characteristic of a distribution with non-finite mean.

You're also correct that this same logic would apply to higher moments - replace $x$ with $x^k$ in your integrals and you can make an exactly analogous argument. Where it gets more complicated is when, for example, both integrals are non-finite but this seems beyond the scope of the question :)

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Mathematically that makes perfect sense, but I still find it a little bit counter-intuitive. For instance, consider $p=0.99$ for a t-distribution with 100 degrees of freedom mixed with a t-distribution with 1 degree of freedom. The resulting pdf definitely looks like it would produce a finite expected value. $\endgroup$ – John Jun 29 '12 at 18:54
  • 1
    $\begingroup$ @John, if you were to generate data from that distribution, one way to do that would be to have $99\%$ of it from a $t_{100}$ distribution and $1%$ of it from a $t_1$ distribution. Well, that $1\%$ will be extremely long-tailed and, as such, will occasionally produce extreme outcomes, leading to the perhaps counterintuitive result that the variable's mean doesn't exist. It's not always clear by looking at a plot of the density function that the mean will not exist - for example, the difference between a $t_1$ and $t_2$ density is subtle but one has a mean while the other doesn't! $\endgroup$ – Macro Jun 29 '12 at 19:01
  • $\begingroup$ @Marco True enough regarding the subtlety of the densities. It isn't particularly challenging to simulate from the distribution that I mentioned. The problem is that mathematically the expected value is undefined, but it is possible to calculate an expected value of the simulated data that is technically incorrect. $\endgroup$ – John Jun 29 '12 at 19:17
  • 1
    $\begingroup$ @Macro I am not convinced by your answer. In fact you start with John's equality $\int_{-\infty}^{\infty}xf_{3}(x)dx=p\int_{-\infty}^{\infty}xf_{1}(x)dx+(1-p)\int_{-\infty}^{\infty}xf_{2}(x)dx$ but $xf_1(x)$ and $xf_2(x)$ are not positive functions hence this equality is not straightforward. If one of the integral is undefinite then this equality has no sense, and if one of the integral is infinite then this equality remains to be proved. $\endgroup$ – Stéphane Laurent Jun 29 '12 at 19:54
  • 1
    $\begingroup$ Ooops sorry ! I haven't seen that the original question assumes that $E(X_1)$ is defined ! $\endgroup$ – Stéphane Laurent Jun 29 '12 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.