The R function binom.test performs an exact binomial test. In trying to understand how many trails would be necessary to reject the null given zero successes, I plotted the p-value as follows

p <- 1/16
n <- seq(2,200)
pval <- lapply(n, function(x) binom.test(0,x,p)$p.value)
plot(n, pval)

plot of pval vs n

I was surprised to see that the p-value did not decrease monotonically with increasing trials. What is the reason for this and is it accurate?

  • 3
    $\begingroup$ It is because the binomial random variable is discrete. See my paper in the American Statistician coauthored with Christine Liu. "The Saw-toothed Behavior of Power vs Sample Size and Software Solutions: Single Binomial Proportions using Exact Methods. American Statistician (2002) Vol. 56 pp 149-155. $\endgroup$ Mar 15, 2018 at 0:25
  • $\begingroup$ I took @StephanKolassa advice. I realize that the result is counter-intuitive and it may take more detailed accounts in the references to get a clear picture. $\endgroup$ Mar 15, 2018 at 14:31

1 Answer 1


The reason is because the binomial random variable is discrete. My paper in The American Statistician [1] explains this and gives other references. Think about the fact that the planned type 1 error cannot be achieved exactly for all values of the sample size n.

[1] Michael R Chernick & Christine Y Liu (2012). The Saw-Toothed Behavior of Power Versus Sample Size and Software Solutions. The American Statistician, 56:2, 149-155, DOI: 10.1198/000313002317572835


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.