2
$\begingroup$

I have a question that requires to prove if the following function whether is it a PMF with poisson random variable. The function is as follows...

$f(x) = \pi \frac {\lambda_1^x}{x!} e^{-\lambda_1} + (1-\pi) \frac {\lambda_2^x}{x!}e^{-\lambda_2} $

where $x\epsilon \mathrm X = \{0,1,...\}, \pi \epsilon (0,1), \lambda_1, \lambda_2 > 0, \lambda_1 \neq \lambda_2$

Can I split $f(x)$ into 2 function $\sum_{x=0}^{\infty}\pi \frac {\lambda_1^x}{x!} e^{-\lambda_1}$ and $\sum_{x=0}^{\infty} (1-\pi) \frac {\lambda_2^x}{x!}e^{-\lambda_2}$ to prove that it is a PMF and hence continue with calculating it's expectation and variance using this 2 sub-function?

I've found out the expectation of $f(x)$, which is $\mathbb E[X] = \mathbb E[X_1] + \mathbb E[X_2];$ where, $\mathbb E[X_1] = \sum_{x=0}^{\infty}x\pi \frac{\lambda_1^x}{x!}e^{-\lambda_1} = \pi\lambda_1$
$\mathbb E[X_2]=\sum_{x=0}^{\infty} x(1-\pi)\frac{\lambda_2^x}{x!}e^{-\lambda_2} = (1-\pi)\lambda_2$

Now my question is how am I suppose to carry to find out what is the variance as the variance i've got is different from the expectation I got above. For a Poisson random variable, the expectation should be the same as variance. Am I right? I have also tried using moment generating function. For the 1st differential, I got the same result as my expectation. But I couldn't further differentiate the 2nd time to get my variance as all the remaining terms are constant terms. Could I have some suggestion of how should I carry on from where?

$\endgroup$
  • $\begingroup$ Why not? What problem(s) do you run into when you carry out the calculations? $\endgroup$ – whuber Sep 11 '13 at 17:06
  • 2
    $\begingroup$ For this $f(x)$ to be a pmf, each of $f(0), f(1), f(2), \ldots$ must be nonnegative (true by inspection here) and the sum $\sum_{i=0}^\infty f(i)$ must equal $1$. If you know the series $$e^\lambda = \sum_{i=0}^\infty \frac{\lambda^i}{i!},$$ you can apply this to $\sum_i f(i)$ to show that $f(\cdot)$ is indeed a pmf (provided that $\pi \in [0,1]$) but that it is not a Poisson pmf except when $\pi = 0$ or $\pi = 1$. $\endgroup$ – Dilip Sarwate Sep 11 '13 at 18:12
  • $\begingroup$ You may be interested in (Eisenberger 1971). Reference:Eisenberger, I. "Estimating the Parameters of the Distribution of a Mixture of Two Poisson Populations." Deep Space Network Progress Report 3 (1971): 94-97. $\endgroup$ – Avraham Sep 11 '13 at 18:18
  • $\begingroup$ Although your edit changes the question, the edit is already answered here. Note that the variance is not a non-central moment of the distribution, but is an algebraic combination of such moments. $\endgroup$ – whuber Sep 12 '13 at 5:41
  • 1
    $\begingroup$ Your calculation of $E[X]$ results in the correct value, but the meanings you ascribe to various parts of your answer are not correct. What you call $E[X_i]$ is not the expectation of any random variable, and certainly not the expectation of a Poisson random variable with parameter $\lambda_i$. Nor is $X$ equal to the sum $X_1+X_2$ from which the relationship $E[X]=E[X_1]+E[X_2]$ that you state would naturally follow. If you will correct these misconceptions (see whuber's answer for the correct formulation of the relationship between $X$ and the $X_i$), you will make more progress. $\endgroup$ – Dilip Sarwate Sep 12 '13 at 13:20
2
$\begingroup$

As per whuber's wishes, I am expanding my comment on the OP's question into a full-fledged answer. Whether the answer is using statistical ideas only or not is a matter for the readership to judge.

Let $\pi \in [0,1]$. Then the function $$f(x) = \begin{cases}(1-\pi)e^{-\lambda_1}\frac{\lambda_1^n}{n!} + \pi e^{-\lambda_2}\frac{\lambda_2^n}{n!}, &x = \text{nonnegative integer}~ n,\\ 0, &\text{otherwise,} \end{cases}$$ is a probability mass function since $f(x) \geq 0$ for all $x$ and $$\begin{align}\sum_{n=0}^\infty (1-\pi)e^{-\lambda_1}\frac{\lambda_1^n}{n!} + \pi e^{-\lambda_2}\frac{\lambda_2^n}{n!} &= (1-\pi)\sum_{n=0}^\infty e^{-\lambda_1}\frac{\lambda_1^n}{n!} + \pi \sum_{n=0}^\infty e^{-\lambda_2}\frac{\lambda_2^n}{n!}\\ &= (1-\pi) + \pi\\ &= 1 \end{align}$$ since the two summands on the right are recognizable as the pmfs of Poisson random variables. The pmf $f(x)$ is called a mixture pmf (of two Poisson random variables). Let $X$ denote the random variable with this mixture pmf. Then, $X$ is not a Poisson pmf except for the extremal value $\pi=0$ and $\pi=1$ in which case we get $X$ is Poisson with parameters $\lambda_1$ and $\lambda_2$ respectively. A proof of the assertion of non-Poissonity is as follows.

Note that $E[X]= (1-\pi)\lambda_1 + \pi \lambda_2$ wnich can be obtained by straightforward summations just as in computing the mean of a Poisson random variable. Now suppose that $X$ is indeed a Poisson random variable with parameter $\lambda$. So we have $$E[X] = \lambda = (1-\pi)\lambda_1 + \pi \lambda_2$$ which, as a function of $\pi$ varies linearly from $\lambda_1$ at $\pi=0$ to $\lambda_2$ at $\pi = 1$. Now, according to the mixture pmf, $$P(X=0) = (1-\pi)e^{-\lambda_1} + \pi e^{-\lambda_2}\tag{1}$$ whereas the alleged Poissonity of $X$ gives us that $$P(X=0) = e^{-\lambda} = e^{-((1-\pi)\lambda_1 + \pi \lambda_2)}.\tag{2}$$ The right sides of these two expressions are not equal and so the assumption that $X$ is a Poisson random variable is not tenable. How do we know that the right sides of $(1)$ and $(2)$ are not equal? Let $Y$ be a discrete random variable taking on values $\lambda_1$ and $\lambda_2$ with probabilities $(1-\pi)$ and $\pi$ respectively. Then $$E[Y] = (1-\pi)\lambda_1 + \pi \lambda_2$$ and so, for the convex function $e^{-x}$ we have, by Jensen's inequality that $$e^{-E[Y]} = e^{-((1-\pi)\lambda_1 + \pi \lambda_2)} \leq E[e^{-Y}] = (1-\pi)e^{-\lambda_1} + \pi e^{-\lambda_2}$$ with equality occurring only at the end points because the straight line through the points $(0,e^{-\lambda_1})$ and $(1,e^{-\lambda_2})$ is strictly above the curve $e^{-((1-\pi)\lambda_1 + \pi \lambda_2)}$ for $\pi \in (0,1)$. That is, the right side of $(2)$ is smaller than the right side of $(1)$ except when $\pi=0$ or $\pi = 1$, and so $X$ is not a Poisson random variable except in these extreme cases.


Note added in response to OP's comment and query

Since $X$ is not a Poisson random variable, it is not necessary that its mean equal its variance as is the case for Poisson random variables. The variance of $X$ can be calculated most easily as indicated in whuber's answer. Begin with the fact that for a Poisson random variable $W$ with parameter $\mu$ (and hence mean $E[W] = \mu$), $$E[W^2] = \sum_{n=0}^\infty n^2e^{-\mu}\frac{\mu^n}{n!} = \mu^2+\mu.$$ and so $$\begin{align} E[X^2] &= \sum_{n=0}^\infty n^2\left[(1-\pi)e^{-\lambda_1}\frac{\lambda_1^n}{n!} + \pi e^{-\lambda_2}\frac{\lambda_2^n}{n!}\right]\\ &= (1-\pi)\sum_{n=0}^\infty n^2e^{-\lambda_1}\frac{\lambda_1^n}{n!} + \pi\sum_{n=0}^\infty n^2e^{-\lambda_2}\frac{\lambda_2^n}{n!}\\ &= (1-\pi)(\lambda_1^2 + \lambda_1) + \pi(\lambda_2^2 +\lambda_2) \end{align}$$ and so $$\begin{align} \text{var}(X) &= E[X^2] - (E[X])^2\\ &= (1-\pi)(\lambda_1^2 + \lambda_1) + \pi(\lambda_2^2 +\lambda_2)- ((1-\pi)\lambda_1 + \pi \lambda_2)^2\\ &= [(1-\pi)\lambda_1 + \pi\lambda_2] + (1-\pi)\lambda_1^2 + \pi\lambda_2^2 - ((1-\pi)\lambda_1 + \pi \lambda_2)^2\\ &= [(1-\pi)\lambda_1 + \pi\lambda_2] + \pi(1-\pi)(\lambda_1-\lambda_2)^2 \end{align}$$ as already pointed out to you by whuber.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Dear Dilip, I did the same to prove f(x) is a PMF. But how m I suppose to find out the $\mathbb Var[X]$ as the result I got is different from with the $\mathbb E[X]$ $\endgroup$ – melyong Sep 12 '13 at 4:23
  • $\begingroup$ For the last part ($var(x)$) how did u get from step 3 to 4? Also did $\lambda$ 1 and 2 got swapped? For me, $\lambda_2$ is the one with the $(1-\pi)$ term $\endgroup$ – Jiew Meng Sep 16 '13 at 14:25
5
$\begingroup$

The left hand side, $f$, is a convex mixture of two probability mass functions (pmf), whence it is a pmf. However, because its variance exceeds its mean, it cannot be the pmf of a Poisson distribution.


Consider a bivariate random variable $(U, X)$ where $U$ is a Bernoulli$(\pi)$ variable and, conditional on the value of $U$, $X$ either has a distribution with probability mass function $f_0$ when $U=0$ or a distribution with pmf $f_1$ when $U=1$. The marginal distribution of $X$ is a mixture of $f_0$ and $f_1$. Let $f$ be the the probability mass function of the marginal distribution and let $x$ be a possible outcome. According to the definitions of pmf and marginal distributions,

$$\eqalign{ f(x) &= \Pr(X=x) \\ &= \Pr(X=x|U=0)\Pr(U=0) + \Pr(X=x|U=1)\Pr(U=1) \\ &= (1-\pi)f_0(x) + \pi f_1(x). }$$

In the question, $f_0$ is the pmf of a Poisson$(\lambda_1)$ distribution and $f_1$ is the pmf of a Poisson$(\lambda_2)$ distribution. Therefore $f$ is a valid pmf.

Is $f$ the pmf of some Poisson distribution, say with parameter $\lambda$? There are many ways to check. Because a Poisson distribution depends on a single parameter, whenever we obtain two numerical properties of the distribution they must have a definite relationship. The best-known relationship in a Poisson distribution is that the variance equals the mean. The first two moments of the mixture are

$$\mu_1 = (1-\pi)\lambda_1 + \pi\lambda_2$$

and

$$\mu_2 = (1-\pi)(\lambda_1+\lambda_1^2) + \pi(\lambda_2 +\lambda_2^2).$$

From these we discover that the variance of the mixture is

$$\mu_2 - \mu_1^2 = \mu_1 + \pi(1-\pi)(\lambda_1-\lambda_2)^2.$$

The right hand side is strictly greater than the mean $\mu_1$ for $0\lt\pi\lt 1$ and $\lambda_1\ne\lambda_2$ because under these conditions $\pi\gt 0,$ $1-\pi\gt 0$, and $(\lambda_1-\lambda_2)^2\gt 0,$ whence the excess over $\mu_1$ on the right hand side is strictly positive. Therefore the mixture cannot be Poisson (it is "over dispersed").

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Anyone unsure about this calculation of the variance of a mixture can see it worked it out in gory detail at stats.stackexchange.com/questions/69399/…. The basic idea is very simple, though, as explained in an answer at stats.stackexchange.com/questions/32699. $\endgroup$ – whuber Sep 11 '13 at 20:54
  • $\begingroup$ As I said in my comment on the question, the mixture cannot be Poisson except in the trivial cases $\pi=0$ or $\pi = 1$, when it is really not a mixture at all but just one of the constituent Poisson random variables. $\endgroup$ – Dilip Sarwate Sep 11 '13 at 21:19
  • 1
    $\begingroup$ @Dilip Yes, you said correctly and I would have been glad to see your comment (more fully fleshed out) posted as an answer. For the record, so that we have a valid answer in this thread, here I have proven the result and explained (using statistical ideas) how one would go about finding such a proof in the first place. $\endgroup$ – whuber Sep 11 '13 at 21:22
  • $\begingroup$ I fleshed out my comment into an answer. $\endgroup$ – Dilip Sarwate Sep 12 '13 at 12:53
0
$\begingroup$

The sum of Poisson Random Variables is a Poisson (http://www.proofwiki.org/wiki/Sum_of_Independent_Poisson_Random_Variables_is_Poisson). Hence you can prove they are individually Poissons to show that their sum is a Poisson.

Expectation: E[A+B] = E[A] + E[B] where A,B are R.V.s

Variance: Var[A+B] = Var[A] + Var[B] + 2Cov(A,B)

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ Unfortunately, this answer is not relevant to the question asked which is about a mixture distribution (weighted sum of two Poisson pmfs with weights summing to $1$) and not about the sum of two Poisson random variables. $\endgroup$ – Dilip Sarwate Sep 11 '13 at 18:05
  • $\begingroup$ Sorry missed that point $\endgroup$ – Budhapest Sep 11 '13 at 18:09
0
$\begingroup$

Probably simply no, you cannot split the pieces and treat both as Poisson, as the two individual pieces:

  1. Do not sum to 1 so are not valid PMFs (may be neither relevant nor true)
  2. Are dependent on each other through $\pi$

For further reading, perhaps see the Eisenberger paper mentioned in the comments on the question and (Wang et al 2006).

Edit

@whuber's answer is far better, as usual.


Reference:

Kui Wang, Kelvin K.W. Yau, Andy H. Lee, Geoffrey J. McLachlan, Two-component Poisson mixture regression modelling of count data with bivariate random effects, Mathematical and Computer Modelling, Volume 46, Issues 11–12, December 2007, Pages 1468-1476, ISSN 0895-7177, http://dx.doi.org/10.1016/j.mcm.2007.02.003. (http://www.sciencedirect.com/science/article/pii/S0895717707001100)

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ What do you mean by the "individual pieces"? The pieces I see in the question are Poisson pmfs; there's nothing invalid about them. $f$ describes a mixture of two Poisson variates (provided $0\lt\pi\lt 1$). $\endgroup$ – whuber Sep 11 '13 at 19:57
  • $\begingroup$ If $\sum_{x=0}^\infty \frac{\lambda^xe^{-\lambda}}{x!} = 1$ then $(1-\pi)\sum_{x=0}^\infty \frac{\lambda^xe^{-\lambda}}{x!} \ne 1$. Each piece without the weight is certainly Poisson, but with the weight the PMF <> 1, and the two are not independent since $\pi$ appears in both, no? Please correct me if I have made an error. $\endgroup$ – Avraham Sep 11 '13 at 20:00
  • $\begingroup$ That strikes me as being less than relevant. What matters here is that $\sum_{x=0}^\infty \frac{\lambda^xe^{-\lambda}}{x!}$ describes a pmf. Thus the question concerns whether a convex linear combination of two pmfs is a pmf. What is ambiguous is whether the OP is also asking whether such a mixture of Poisson distributions can be Poisson; I suspect they are not asking this but it might seem they are because of difficulties with English. $\endgroup$ – whuber Sep 11 '13 at 20:02
  • 1
    $\begingroup$ @whuber OK, now we're swimming way beyond my capabilities and understanding, but per this entry a convex linear combination of characteristic functions is also a characteristic function. Since there is a one-to-one relationship between characteristic functions and CDFs, this would mean that the convex linear combination does define a PMF. There is no guarantee that PMF is Poisson, though. $\endgroup$ – Avraham Sep 11 '13 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.