Standard deviation of a scaled Poisson process

Question

Assume we have some Poisson process that produces events. In a given year we have counted $N$ of these events.

Further assume that for some reason we need to report a monthly rate instead of this yearly number and also the (estimated) standard deviation in this monthly rate.

Clearly, the monthly rate is $N/12$. Now, the question is: What is the standard deviation in this monthly number? We have two contradicting views on this.

Alice maintains that, since the monthly number ($X$) is just a scaled version of the yearly figure ($Y$), one could just apply the scaling rule for variances.

Then, with $X = Y/12$ it follows that $\rm{Var}(X) = \frac{1}{12^2}\rm{Var}(Y)$ and hence the standard deviation of the monthly figure is 1/12 of the standard deviation of the yearly figure. The latter standard deviation is $\sqrt{N}$ as this is a Poisson process. So, we have $\sigma_{X}=\sqrt{N}/12$.

Bob, on the other hand, argues that the results for each month are generated by a Poisson process with a parameter that is scaled by 12. This follows from the rule w.r.t. the sums of Poisson distributed variables. So, with $Y\sim \rm{Pois(N)}$ it follows that $X\sim \rm{Pois(N/12)}$. Clearly, $\sigma_{X}$ is just the standard deviation of such a Poisson process, which is the square root of its rate parameter. Therefore, $\sigma_{X}=\sqrt{N/12}$.

Although the means resulting from Alice's and Bob's reasoning are the same, we've got a factor of $\sqrt{12}$ between their respective standard deviations. Who is right here, Alice or Bob?

Note: The standard deviation of this monthly number is to be understood as the (theoretical) standard deviation of future determinations of this monthly number generated by the same, assumed Poisson process.

Answer 1

It's important to be very specific about what is going on. Your description is somewhat ambiguous, which is why you're confused.

If you observe a homogeneous Poisson process whose annual event rate is $\lambda$ for one month (pretending all months are the same length, which they really aren't), the number of events per month would be Poisson with rate $\lambda/12$. The population variance of that month's number of events would be $\lambda/12$.

Let $N$ be the number of events in a homogeneous Poisson process in a year. If you observe $n$ events, then $\hat{\lambda}=n$ and $\widehat{\text{Var}}(\hat{\lambda})=n$.

Let the number of events in a specific future one twelfth of a year be $M$. Then let $\widehat{M}$ be your predicted number of events in that "month" using the obvious estimator ($\widehat{M}=n/12$).

(i) If you're estimating the Poisson parameter for a future month, your estimate of the variance of that parameter estimate is (n/144). $\text{Var}(\widehat{M}|N=n)=n/144$.

(ii) If you're predicting the number of events in a future month, your estimate of the variance in the observed number of events in the future month will be the predicted Poisson rate (n/12). That is $\widehat{\text{Var}}(M|N=n)=n/12$.

However, if you're interested in how far out your prediction of that future count might be, you will want to estimate the variance of the prediction error $\widehat{\text{Var}}(M-\hat{M}|N=n)$, which is the sum of the two ($n/12+n/144$).

(Indeed, compare with the law of total variance, where the term in (i) is the variance of a conditional expectation and the term in (ii) is in effect the expectation of a conditional variance.)

Answer 2

You have for the number of counts:

• Counts per year: $$Y \sim Pois(\lambda)$$
• Counts per month: $$X \sim Pois(\lambda/12)$$

But...

• Counts per month (average for 12 months) $$Y/12 \nsim Pois(\lambda/12)$$ or $$\frac{X_1+X_2+...X_{12}}{12} \nsim Pois(\lambda/12) $$

If you divide the counts over a year by twelve then you do not get a variable that corresponds to the counts for a particular individual month, but instead you get an average over twelve months.

---

The Poisson distribution is only to be used for the raw number of counts. It is not true for any derived (scaled) number. So a term like 'counts per T' should be used very carefully. The Poisson distribution describes 'counts' and not 'counts per T'.

---

Bob was right in stating that $\text{Var}(X) = \frac{\lambda}{12}$. However when you take the mean of twelve of those variables (which is what Alice computed) then you will get:

$$\text{Var} \left( \frac{1}{12} (X_1+X_2+...X_{12}) \right) = \frac{1}{12}\frac{\lambda}{12} = \frac{\lambda}{12^2}$$

and the standard deviation, $\sigma = \frac{\sqrt{\lambda}}{12}$, corresponds to Alices number.

---

for some reason we need to report a monthly rate instead of this yearly number and also the (estimated) standard deviation in this monthly rate

You can report

• an estimate for the mean monthly rate and the estimated standard error for that estimate.

But note that this will be different from

• the standard deviation of that monthly rate.

The variance of a distribution and the variance of an estimate for the mean of that distribution are not the same.

(This occurs very often that some people report figures with very tiny error bars. That makes it appear as if the difference between two cases is very small. But, what those people only did is show that they can estimate the means very precise and show those are different, but this does not mean that the differences between the groups is so large. Often it is also confusing/ambiguous what the reported variation/error bars mean.)

Answer 3

what I understand is that you know just the number $N_{\rm y}$ of events that occurred in a year time interval $t_{\rm y}$. From that, you would like to report an estimation of how many events $N_{\rm m}$ occurred in each month time interval $t_{\rm m}$, and importantly, what is the variability that one would expect for that estimated $N_{\rm m}$.

Answer: The approach would be to estimate the rate $\lambda$ by its maximum likelihood estimator, $\hat{\lambda} = N_{\rm y}/t_{\rm y}$. If we were talking about a prediction for next month, then assuming that the rate parameter $\lambda$ does not depend on time you can use $\hat{\lambda}$ and obtain

$$ \left< N_{\rm m} \right> = N_{\rm y}/12 $$ $$ \textrm{std}(N_{\rm m}) = \sqrt{ N_{\rm y}/12 } $$

However, it doesn't matter if we are dealing with a prediction for next month or a prediction of what already happened but you don't know, so this result is the one you are looking for.

Note: if you have a prior knowledge for reasonable values of $\lambda$, you could go further and use Bayesian statistics, which actually modify the form of $\textrm{std}(N_{\rm m})$.

Why the transformation of variables is not correct? Because in that case you are not dealing with the statistics of the monthly results, you are still dealing with the statistics of a yearly result but you consider that each individual event which counts, instead of as one unit $N=1$, as $N=1/12$. In that case you get estimations of the same order for the average, that's clear, but the variability is not the right one for a month.

Hope it helps.