# How can I do a hypothesis test with an estimated null hypothesis?

The situation I have is with data that (I'm assuming for now) follows a poisson distribution. I want to know if this year had an unusual number of a certain type of fatality compared to previous years. I could use a hypothesis test to evaluate if this result is unusual. If the average rate in previous years was 8 fatalities per year, and last year there were 13 fatalities:

Null hypothesis: λ = 8 Alternative hypothesis: λ > 8 Using a cumulative poisson distribution table, I can see that P[x>=13] = 0.0342 when λ = 8 is, so the result is statistically significant at the 0.05 level.

However, I only have a few years of data with which to estimate my null hypothesis λ = 8. I know that the standard error for my estimate is sqrt(λ/n) - how do I include this extra uncertainty in my hypothesis test?

Thanks!

Edit: Thanks everyone for your responses so far. Some have asked for specifics of the data I am looking at. I actually want to stay away from solutions for specific data since this is a business question that comes up a lot in with different types of accidents and with different numbers of years available, and I'm looking for a good general solution. If it helps, lets say I always have between 7 and 15 years of data to give me my null hypothesis lambda.

This is a difficult situation with not much data, but I think your analysis is reasonable. However, it seems you have a computational error.

Suppose $$40$$ accidents of a particular kind over the last $$5$$ years. (The annual numbers may have been 7. 11, 5, 9, 8.) It seems reasonable to model these accidents according to a Poisson distribution and the estimated Poisson mean is $$\hat \lambda = 40/5 = 8.$$

Now in the current year just finished, there have been $$13$$ accidents of this kind. It is certainly possible that one of the previous five years might have had $$13$$ accidents. The annual numbers might have been 7, 8, 13, 9, 3. Generally speaking, it is certainly possible for a Poisson distribution to produce an occasional large count.

How likely is that? The result from R is $$P(X \ge 13) = 1 - P(X \le 12) = 0.064.$$ So the P-value of your test is about 6.4% > 5% and you cannot reject $$H_0: \lambda = 8$$ in favor of $$H_a: \lambda > 8$$ at the 5% level of significance.

1 - ppois(12, 8)
[1] 0.0637972


If the count for the most recent year had been $$14$$ then the P-value would be about 3.4%, leading to rejection at the 5% level.

1 - ppois(13, 8)
[1] 0.0341807


Notes: With so little data and with 'borderline' P-values, subjective factors are bound to affect the believability of this test:

(a) In order for this test procedure to work, it is necessary for the correct model to be Poisson with the same mean $$\lambda$$ holding true throughout the 5-year reference period. With only five years of data for reference, there is no way to check whether the Poisson model is correct or whether $$\lambda$$ has been constant.

If the numbers for the past five years were something like 5, 7, 9, 8, 11, then I would worry that there might be an increasing accident rate across all six years. In that case it wouldn't be surprising for the sixth year to be larger.

(b) If something occurred just before the beginning of the last year (with its 13 accidents) that might be expected to increase the accident rate, and that occurrence triggered doing the test, then I would be more inclined to trust the test. (E.g., heavier traffic on a road due to a new factory nearby, or major construction with narrower traffic lanes at unpredictable places throughout the year.)

By contrast, if you just notice a year with 13 accidents and decide to do a test based on that alone, I would be less inclined to trust the test.

• Although your caveats are well considered, I believe they don't validate your approach. Any comparison of this year's result to the previous years' results ought to handle the uncertainties in all six years of data appropriately, rather than assuming there is no uncertainty in the first five. Thus, given there is an unknown underlying distribution common to all six years, what evidence against this does the value $(40,13)$ of the random variable $(X,Y)$ given by the sum of five years and the sixth year provide? Using $Y-X/5$ as test statistic, $p\approx 11.5\%$ isn't significant. – whuber Mar 31 '19 at 22:23
• Another way to look at this is to examine the distribution of the largest value out of six independent draws from a Poisson$((40+13)/6)$ distribution: $13$ is its median value! (The chance of $13$ or larger is $51\%.$) Thus, we should expect to see a value of $13$ in six years. Even the appearance of the $13$ in a specific year of interest (the latest) is still no real surprise. – whuber Mar 31 '19 at 22:28
• @whuber: OP said 'previous years' and I think it was a bad idea to take that to be 5. I understand the argument about looking at the max. However, I'm confused about median of POIS(53/6) being 13: from qpois(.5, 53/6) I get 9. Anyhow, it seems neither of the two Answers works well, and I'm willing to delete mine. Do you think there is any good Answer to the original question? – BruceET Apr 1 '19 at 0:10
• The median of the maximum of six iid observations is 13. – whuber Apr 1 '19 at 3:24
• @whuber your first comment gets at exactly my problem: all examples I see of hypothesis tests use some "known" value of lambda - but I don't have much confidence in the only value for lambda I have available, which is the average for the past 7-15 years. – KDford Apr 1 '19 at 14:19

My first-nanosecond, Malcolm-Gladwell-Blink-level reaction asks why you're using the cumulative Poisson as the distribution of your test statistic. If you've got a small number of samples from any distribution (including the Poisson), the Central Limit Theorem says you can use the t- or F- distribution (via a t-test or an ANOVA) to test your hypothesis. The small sample size would be worked into your test in computing the degrees of freedom.

So, H0: λ = 8, HA: λ ≠ 0, and since you have a small sample ("a few data points"), compute a t-statistic with df = a few − 1.

If there's a reason to use a Poisson CDF, I'm not seeing it at the moment. Tell us more, or give us more data or more context.

• Could you please explain how the CLT might apply to a "small number of samples" from "any distribution"? Both quoted phrases appear to preclude application of the CLT. – whuber Mar 31 '19 at 22:13
• You’re absolutely right about “a few”. It depends on the underlying distribution. I don’t know how many data points he has, and tried to qualify it. The CLT is powerful, so without looking at the data, I couldn’t say specifically. There are always nonparametric/distribution free methods available. If the original poster followed up, that would have been my next suggestion. – Sciolism Apparently Mar 31 '19 at 22:26
• @whuber Also, the CLT says that any—really, any—original distribution, will produce sample distributions that approach normality. I feel sure you know how to do a simulation: start with something completely non-normal, like, say, a Chi-Squared distribution with 1df. Randomly sample 5 values, average them, and plot the average. Do the same thing again. And again. By the time you’re around 20–30, the sampling distribution will look Normal. My point was that I couldn’t imagine using Poisson as the distribution of a sampling stat. – Sciolism Apparently Mar 31 '19 at 22:37
• Please see stats.stackexchange.com/questions/69898/… for one (real world) reason why your assertions are not true. In the present case, since there is exactly one observation in year 6, the CLT is utterly silent about how it might reasonably be compared to the sum for years 1-5. – whuber Mar 31 '19 at 22:40
• @whuber The only information that existed in the thread when I responded was that the original data followed a Poisson distribution, with an average of 8 fatalities for an unknown number of years, and that last year there were 13 fatalities. How many are in each year I have no idea. Certainly, years with zero or one data point are special cases. But that kind of specificity wasn’t present in the original question or my response. Note my final sentence asking for more info, more data, more context. – Sciolism Apparently Mar 31 '19 at 23:13