The book "Testing Statistical Hypotheses" by Lehman describes in section 4.5, a test for comparing the rates of two Poisson point-processes. This is the "uniformly most powerful" (UMP) test for this process. Now, a data set I was looking at had higher variance than mean which violates the assumption of the count of events in an interval being Poisson distributed (which has the same variance as mean). So, I decided that a Compound Poisson process might be a good fit for such data. What this means is that the arrivals are still governed by a Poisson process, but each time there is an arrival, another random variable governs the number of point-events (instead of one event per arrival). The distribution I considered for the distribution of the number of point events with each arrival was Binomial.

Now, I was expecting the power of the UMP Poisson test to be compromised if used on this Compound Binomial process. So, I ran some simulations generating first from a null hypothesis (two identical distributions) and then alternate (Poisson arrival rate increased by some effect size) to get the false positive and false negative rates ($\alpha$ and $\beta$ respectively). The $\alpha$-$\beta$ curve I thought would be "better" (less $\beta$ for a given $\alpha$) when I simulated from a Poisson than when I used a Compound Poisson. However, as the figure below shows, this didn't seem to be the case (note only the blue and orange curves which are on top of each other). It seems the power of the test stays the same, which is very surprising. Note that the x-axis on the first graph is the actual $\alpha$ you get from the test (which might be different from the $\alpha$ you compare the p-value to if your distributional assumption is violated).

enter image description here

The only consequence of using the UMP Poisson test on a Compound Poisson seems to be that the $\alpha$ you set differs from the $\alpha$ you get. The relationship is shown in the figure below. But apart from that, the power isn't compromised. Does anyone have some intuition or a proof as to why this might be?

enter image description here


2 Answers 2


[I can't comment on your answer as I don't have the min reputation needed for it.]

Poisson case: You know the UMP test here. That is, given the observations of two Poisson processes - arrival times $\{t_i^{(p)}\}_{i=1}^{n_p}$ for processes $p=1,2$ - you have a function $f$ such that, if $f(\{t_i^{(1)}\}_{i=1}^{n_1}, \{t_i^{(2)}\}_{i=1}^{n_2}) > 0$ then you reject the null hypothesis.

Compound Poisson case: Here you observe $\{(t_i^{(p)}, e_i^{(p)})\}_{i=1}^{n_p}$ for processes p=1,2, where $t$ denotes the arrival times and $e$ the number of events. Let $\lambda$ be the arrival rate, which influences $T$ but not $E$ ($E$ and $T$ are independent by assumption). Further, by assumption, the only thing distinguishing the two processes is $T_1$ and $T_2$ (via $\lambda_1$ and $\lambda_2$), since $E_p,\, p=1,2,$ are identically distributed and not influenced by $T_p,\, p=1,2$. Thus, only considering $\{t_i^{(p)}\}_{i=1}^{n_p}$ for $p=1,2$ is sufficient for any hypothesis test. But since $T_p,\, p=1,2$, are Poisson distributed as before, the same test remains optimal.


I have an outline of a proof. The outline has some jumps, but these jumps will be rectified in time.

I begin with the observation that if I take the binomial test and scale $n_1$ and $n_2$, the $\hat{\alpha}$ ($\alpha$ you set) vs $\alpha$ (False Positive Rate you get) graph starts to straighten out. The figure below is the same to the second figure in the question with one added line (green one). For that one, I scaled both $n_1$ and $n_2$ used in the binomial test down by the average value of $Y$, which was the random variable responsible for the compounding (number of point events each time a Poisson arrival hits). Once I do this, it becomes like I was applying the UMP Poisson test on the Poisson random variable (since the green line becomes close to a straight line and nearly coincides with the blue one). I use this observation to conclude that:

Conjecture - 1: Applying the UMP Poisson test to the compound Poisson with the compounding process given be the random variable, $Y$ is the same as applying an adjusted UMP Poisson test with $n_1$ and $n_2$ scaled down by $E(Y)$ to the Poisson process.

This makes intuitive sense since if we see $n_1$ and $n_2$ events from the two Compound Poissons, the expected number of Poisson arrivals would have been $\frac{n_1}{E(Y)}$ and $\frac{n_2}{E(Y)}$. This is the first logical jump in the proof which I'll fill in later, but let's take it as fact for now.

Now, we only need to prove that applying the modified UMP Poisson where we scale $n_1$ and $n_2$ by some scaling factor, $c$ has the same false negative rate, $\beta$ as the original UMP Poisson when applied to the Poisson process (at the same $\alpha$ or false positive rate we observe).

Suppose the first Poisson process produces $n_1$ point events and the second one, $n_2$ events for a total of $n=n_1+n_2$ total events. These happen in time intervals $t_1=t_2=t$.

When we apply the regular UMP Poisson test, we compare the ratio $\frac{n_1}{n_1+n_2}$ to $p=0.5$. Under the null hypothesis, the number of events from the first process, $n_1$ has the distribution:

$$X_0 \sim \text{Bin}(n,0.5)$$

Under the alternate hypothesis, $n_1$ has the distribution:

$$X_a \sim \text{Bin}(n,\frac{\lambda + \delta \lambda}{2\lambda+\delta \lambda})$$

where $\delta \lambda$ is the effect size on the rate.

When we apply the scaled test, we'll be using under the null hypothesis the random variable:

$$Y_0 \sim \text{Bin}(\frac{n}{c},0.5)$$

And under the alternate hypothesis,

$$Y_a \sim \text{Bin}(\frac{n}{c},\frac{\lambda + \delta \lambda}{2\lambda+\delta \lambda})$$

Now, for the regular, UMP Poisson test, the false positive rate we get conditional on $n$ (assuming the alternate hypothesis being "less" as it makes things a little simpler and assuming we compare the p-value with $\hat{\alpha}$):

$$\alpha_1^{(n)} = P(F_{X_0}(X_0)<\hat{\alpha}) = P(X_0 < F_{X_0}^{-1}(\hat{\alpha})) = F_{X_0}F_{X_0}^{-1}(\hat{\alpha}) = \hat{\alpha}\tag{1}$$

This uses the fact that the distribution of the p-value is given by $F_{X_0}(X_0)$ for why, see here and here.

Similarly, under the alternate hypothesis the false negative rate conditional on $n$:

$$\beta_1^{(n)} = P(F_{X_0}(X_a)>\hat{\alpha}) = P(X_a>F_{X_0}^{-1}(\hat{\alpha})) = S_{X_a}(F_{X_0}^{-1}(\hat{\alpha})) \tag{2}$$

Now, the same numbers of the scaled test:

$$\alpha_2^{(n)} = P(F_{Y_0}(X_0)<\hat{\alpha})=P(X_0<F_{Y_0}^{-1}(\hat{\alpha})) = F_{X_0}F_{Y_0}^{-1}(\hat{\alpha}) \tag{3}$$

$$\beta_2^{(n)} = P(F_{Y_0}(X_a)>\hat{\alpha}) = P(X_a>F_{Y_0}^{-1}(\hat{\alpha}))=S_{X_a}F_{Y_0}^{-1}(\hat{\alpha}) \tag{4}$$

When we set $\hat{\alpha}$, the actual FPR we get from the first test is $\hat{\alpha}$ and that from the second test is: $\alpha = F_{X_0}F_{Y_0}^{-1}(\hat{\alpha})$.

So we need to prove: $\beta_2(\hat{\alpha}) = \beta_1(\alpha)$. Using equations (1) through (4), we get:

$$S_{X_a}F_{Y_0}^{-1}(\hat{\alpha}) = S_{X_a}F_{X_0}^{-1}F_{X_0}F_{Y_0}^{-1}(\hat{\alpha}) = S_{X_a}F_{Y_0}^{-1}(\hat{\alpha})$$

So, LHS=RHS and we've proved the result (sans conjecture-1).

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.