Let $X$ be a Negative Binomial random variable, $X \sim \mbox{NB}(n, p)$, with $P(X=x) = {n+x-1 \choose x} p^n (1-p)^x, x = 0, 1, 2, \cdots$. Suppose I do this experiment where I toss a coin until I got 8 heads. I got the 8th head on the 10th trial. So, we have $n=8$ and $x=2$. Now, I want to test the hypothesis that $p=0.5$ against a two-sided alternative of $p \ne 0.5$. What would be the critical region for this test? How would I calculate a two-sided P-value?

P-value is defined as the probability, under the Null, of getting a result as extreme or more extreme than what was obtained in a particular experiment. Here, it is obvious that $S=\{0,1,2\}$ constitute the critical region of extreme values. Therefore, $P(X \in S) = 0.0547$. However, this is only a one-sided P-value. It is not obvious to me how to define the critical region for the other tail of the asymmetric distribution. For instance, $P(X > 15) = 0.0466 $. Would the two-sided P-value for my experiment be $p=P(S_2)$ where $S_2 = {\{0,1,2\}} \cup {\{16 , \cdots\}}$?

  • $\begingroup$ There are a number of posts on site that address issues relevant to this question (p-values and rejection regions under asymmetric and/or discrete test statistics) $\endgroup$
    – Glen_b
    Commented Oct 9, 2022 at 8:02

1 Answer 1


It depends on how you define your test statistic/how you decide to measure "more extreme"; there's multiple ways you might do so.

I urge you to frame decision rules in terms of choices of test statistics and rejection regions in relation to them first and foremost; many of the difficulties that tend to trip people up disappear, and then bring in p-values at the end. Once you understand what your chosen set of nested rejection regions are, p-values that go with them are a given.

One simple choice for measuring "deviation from the null" that admits a sense of more extreme might be in terms of the deviation from the expected count under $H_0$ (i.e. $|X-E_0(X)|$). That expected value is $8$, so for it the tail values in "$2$ and below" and "$14$ and above" would be "at least as extreme" as the observed $2$.

Of course one is free to choose some other measure of "deviation" from what you would see under the null.

A different choice would be in terms of the likelihood under the null (per Fisher, as we see in the Fisher-Yates-Irwin exact test); . This again gives "$14$" as the cut off in the upper tail in this case, because $p(14)$ is the largest pmf-value that's no larger than $p(2)$; the "more extreme" values are all the cases that are less probable than $p(2)$ - which for a unimodal distribution will be in the tails.

Plot of negative binomial pmf with n=8 and p=0.5 showing 14 is the value with largest probability in the upper tail such that p(x)≤p(2)

larger version

Another approach would be to construct a likelihood ratio statistic, and identify (equal or) smaller likelihood ratios than the one observed ($\Lambda=\hat{\mathcal{L}}_0(x)/\hat{\mathcal{L}}_1(x)$) as more extreme.

Finally, some people simply double the one-sided p-value of the 'nearest' side. (Personally I find this unsatisfying/problematic in several ways, but it's very common.)

While it discusses the continuous case, there's some discussion here that's at least somewhat relevant to this discussion: https://stats.stackexchange.com/a/341442/805

  • $\begingroup$ Thanks @Glen_b. I like the idea of defining the rejection region based on the likelihood ratio statistic $R_k = \{x: \frac{L(x)}{L_{max}(x)} \le k \}$ where $k$ is chosen to obtain the size, e.g., 0.05. Your idea of expected deviation is interesting! I agree that simply doubling one-sided P-value is least satisfying. $\endgroup$
    – user67724
    Commented Oct 9, 2022 at 14:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.