For a two-sided hypothesis test, we need to sum both tails when calculating the p-value. For a test statistic that has a continuous and symmetric distribution like the two-sample t-test, this is quite trivial. It is less trivial (to me at least) for a test statistic with a discrete, asymmetric distribution. Take the binomial test for example. Say our null hypothesis is that the probability of Bernoulli success, $p=0.4$. When I use the binom.test in R, I get the following results:
binom.test(5,10,.4)
meaning I saw 5 heads out of 10 trials, the p-value it returns is:
$$\phi = 1-{10\choose 4}.4^4.6^6-{10 \choose 3}.4^3.6^7=0.5342$$ In other words, from the 5 binomial terms, the ones corresponding to 3 and 4 successes were excluded from the p-value.
Now, what if I observe 3 successes:
binom.test(3,10,.4)
The p-value now returned is: $$\phi = 1-{10\choose 4}.4^4.6^6=0.7492$$ So now, only the term corresponding to 4 successes was excluded from the p-value.
What is the concrete rule to say which terms should be excluded in the two-sided test? Equivalently, is it possible for anyone to point to the actual source code on GitHub? I couldn't locate it.
binom.test
in the console - the relevant part is the assignment of a value to thePVAL
variable. $\endgroup$