# What is the proper test for sparse binomial data?

### I'd like to preface this by saying I am a statistics newbie.

I have a single sample from a binomial distribution of some experiment. The exact numbers are:

Yes: 1

No: 45

I'd like to determine (1) the proper hypothesis test and (2) test statistic for whether the true probability of "yes" is less than 0.05 (not "at most 0.05", if that is different).

# Hypothesis test

It seems I should do a one-tail test. Therefore the hypothesis test is:

$$H_{0}: p < 0.05$$

$$H_{a}: p \geq 0.05$$

Is this correct?

# Test statistic

I had considered a Chi-squared goodness of fit test, but one of the underlying assumptions is that the expected frequency for each category is more than 5. If I treat 0.05 as the theoretical probability, then 0.05 * 44=2.2, which is less than 5 (assuming I'm computing this correctly). I've heard that you can apply Yates correction to Chi-squared but that has its own problems.

So perhaps use an exact one-sided binomial test instead. In this case, can I use the binomial test statistic directly as the probability for my hypothesis test?

A properly formed null hypothesis must always contain an $$=$$-sign (whether as $$=, \le$$ or $$\ge).$$ The equality in the null hypothesis provides the null distribution and thus a probability structure for the test.

So you could test $$H_0: p \le .05$$ against $$H_a: p > .05.$$ In that case, you would reject $$H_0$$ if you had surprisingly many Successes.

Your null distribution for the number of Successes is $$X \sim \mathsf{Binom}(n=46, p=.05).$$ The P-value is the probability of getting the observed number $$X = 1$$ of successes---or more. Specifically, $$P(X \ge 1\,|\,p=0.05) = 1 - P(X = 0\,|\, p-.05)\\ = 1 - (.95)^{46} = 0.9055.$$

This can be computed in R statistical software as:

1 - dbinom(0, 46, .05)
[1] 0.9055318
1 - .95^45
[1] 0.9005597


So you cannot reject $$H_0$$ at the 5% level of significance because the P-value is larger than $$0.05.$$

Perhaps you intended to test $$H_0: p \ge .05$$ vs. $$H_a: p < .05.$$ In that case, the P-value is $$P(X \le 1 \,|\, p=0.0 5) = 0.3232 > .05.$$ So, again you cannot reject this $$H_0$$ at the 5% level.

For reference, here is a plot of the PDF of the distribution $$\mathsf{Binom}(n = 46, p = 0.05).$$ Notice that $$\{X = 1\}$$ is one of the most likely events for this distribution. So if you observe $$X = 1,$$ then the value $$p = .05$$ is consistent with that outcome.

pbinom(1, 46, .05)
[1] 0.3231808


Note: I was going to discuss a confidence interval for this situation, but I see that @Bjorn (+1) has done that while I have been working on the above (with several interruptions).

For a start, it is clear that you will not be able to reject the null hypothesis that p<0.05, because the point estimate is already <0.05.

You might wish to look at an exact (Clopper-Pearson) confidence interval, which would be (0.0006 to 0.1153) for a two-sided 95% CI and (0.0011 to 0.0990) for a two-sided 90% CI. That shows that you would also (with a test based on the CI excluding 0.05) not be able to reject the inverted null hypothesis (of $$H_0:p>0.05$$ vs. $$H_A:p\leq 0.05$$), either, with a one-sided or two-sided level 0.05 test.