1
$\begingroup$

I am playing around with a set of count data $\{k_j\}$, $j=1,...,N$ where all data points are i.i.d. Poisson distributed with (unknown) mean $\mu$. I have tried two ways of estimating $\mu$ from the data and was surprised that they will lead to the same posterior distribution

$$k_j \sim \text{Pois}(\mu)$$

1 Using The Sum of the Data

I know that the sum of all data $K=\sum_{j=1}^N k_j$ is Poisson distributed with mean $N\mu$. That means the likelihood of obtaining the sum $K$ is just

$$P(K\vert \mu,N) = \frac{(N\mu)^K}{K!} \exp(-N\mu).$$

2 Using the Whole Set

Since the results are independent I can write the likelihood of obtaining the set $\{k_j\}$ with given size $N$ and $\mu$ as

$$\begin{eqnarray} P(\{k_j\}\vert \mu, N) &=& \prod_j P(k_j\vert \mu) \\ &=& \frac{\mu^{\sum_j k_j}}{\prod_j k_j!}\exp(-N\mu) \\ &=& \frac{K!}{N^K\cdot\prod_j k_j!}\cdot\frac{(N\mu)^K}{K!}\exp(-N\mu) \\ &=& \frac{K!}{N^K\cdot\prod_j k_j!} P(K \vert \mu , N), \end{eqnarray}$$

where I have used $N=\sum_j 1$ and $K=\sum_j k_j$. What I find interesting is that the likelihood is just the term from the first approach, but multiplied with a constant that does not depend on $\mu$.

So if we want to calculate the posterior distribution via Bayes Theorem, the constant will cancel out because it will appear in the evidence term (marginal likelihood) as well. So given the same prior, both approaches will lead to the same posterior, if I haven't made any errors.

3 My Questions

  1. I was surprised that we can use the sum of the data (and the size of the dataset) to estimate the mean with just as much confidence as if we had kept all data. Does this have something to do with the fact that we want to estimate the mean and not any other parameter?

  2. Furthermore I tried to think if this is true for other distributions. I did a quick and dirty estimation for the same problem if $k_j \sim \text{Normal}(\mu, \sigma)$, with unknown $\mu$ and given $\sigma$. Here I had the same result (though I might have made an error). If the $k_j$ were uniformly distributed then I don't think the result would be true, since the sum is not uniformly distributed any more. Am I right about this?

  3. Is this just a special case of a more general result that I just don't know about?

All hints appreciated. Thank you.

$\endgroup$
7
  • 1
    $\begingroup$ The key is that the population mean is often estimated by the sample mean $\bar X = \frac 1 n \sum_{i=1}^n X_i.$ Also, for $X_1, X_2, \dots, X_n$ IID $\mathsf{Pois}(\lambda),$ one has $T = \sum_{i=1}^n X_i \sim \mathsf{Pois}(n\lambda),$ as you have shown. $\endgroup$
    – BruceET
    May 1, 2020 at 17:12
  • $\begingroup$ @BruceET: Thanks, could you elaborate more on the the "often estimated" statement? Is this due to a more general principle? What properties does the underlying distribution need for that to work? Does it only work if I want to estimate the sample mean or also with other parameters of the distribution? $\endgroup$
    – geo
    May 1, 2020 at 18:21
  • 1
    $\begingroup$ Bernoulli, Poisson, normal, exponential, .... $\endgroup$
    – BruceET
    May 1, 2020 at 19:49
  • $\begingroup$ @BruceET Thank you. Do you have a source where I can read up on this? $\endgroup$
    – geo
    May 1, 2020 at 19:50
  • $\begingroup$ Your difficulty is not clear, so it's difficult to give a link. Maybe this is sufficiently generic. $\endgroup$
    – BruceET
    May 1, 2020 at 20:02

1 Answer 1

2
$\begingroup$

The key point is that $\sum\limits_1^n k_j$ is a sufficient statistic for estimating the parameter $\mu$ of a Poisson distribution.

More precisely, the conditional probability distribution of seeing the data $k_1,k_2,\ldots,k_n$, given that $\sum\limits_1^n k_j=s$ for any non-negative integer $s$, does not depend on the parameter $\mu$. So nothing more about the observations than knowing $\sum\limits_1^n k_j$ (and $n$) can provide additional useful information when you try to estimate $\mu$.

The probability of observing $k_1,k_2,\ldots,k_n$ is $\prod e^{-\mu}\frac{\mu^{k_j}}{k_j!} = e^{-n\mu} \mu^{\sum k_j} \frac{1}{\prod k_j!}$ so you can use the factorisation theorem to show it is a sufficient statistic. The $\frac{1}{\prod k_j!}$ part does not involve $\mu$ so is not helpful in estimating $\mu$, while the other terms simply involve $\mu$ and the sufficient statistic $\sum k_j$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.