# How does the two-sided version binom.test in R work?

For a two-sided hypothesis test, we need to sum both tails when calculating the p-value. For a test statistic that has a continuous and symmetric distribution like the two-sample t-test, this is quite trivial. It is less trivial (to me at least) for a test statistic with a discrete, asymmetric distribution. Take the binomial test for example. Say our null hypothesis is that the probability of Bernoulli success, $$p=0.4$$. When I use the binom.test in R, I get the following results:

binom.test(5,10,.4)


meaning I saw 5 heads out of 10 trials, the p-value it returns is:

$$\phi = 1-{10\choose 4}.4^4.6^6-{10 \choose 3}.4^3.6^7=0.5342$$ In other words, from the 5 binomial terms, the ones corresponding to 3 and 4 successes were excluded from the p-value.

Now, what if I observe 3 successes:

binom.test(3,10,.4)


The p-value now returned is: $$\phi = 1-{10\choose 4}.4^4.6^6=0.7492$$ So now, only the term corresponding to 4 successes was excluded from the p-value.

What is the concrete rule to say which terms should be excluded in the two-sided test? Equivalently, is it possible for anyone to point to the actual source code on GitHub? I couldn't locate it.

• You can inspect the code simply by typing binom.test in the console - the relevant part is the assignment of a value to the PVAL variable. Mar 16, 2020 at 15:53

I think I got the answer on the mechanical level by searching the scipy repo (the function performs equivalently to the R function): https://github.com/scipy/scipy/search?q=binom_test&unscoped_q=binom_test and locating binom_test. They basically take the terms on the other side of the tail which have a PMF lower than the input, $$x$$. In the example given in the question, the PMF of the binomial at $$5$$ is lower than the one at $$3$$. Hence, the term corresponding to $$5$$ is included in the p_Value when $$x=3$$ but the term corresponding to $$3$$ is not included in the p_Value when $$x=5$$. Still looking for some intuition for this.
• (+1) Perhaps edit your q. or ask a new one about why the two-tailed p-value should be defined this way. Especially unintuitive is that this p-value isn't a bimonotone function of the probability postulated by the null hypothesis. For 3 successes in 10 trials, increasing $p$ from 0.40 to 0.41 lowers the p-value, as you'd expect, from 0.7492 to 0.5409; but increasing it to only 0.408 raises the p-value to 0.7495 Mar 16, 2020 at 13:09