2
$\begingroup$

Consider the following example from This binomial test wikipedia article:

Suppose we have a board game that depends on the roll of one die and attaches special importance to rolling a 6. In a particular game, the die is rolled 235 times, and 6 comes up 51 times. If the die is fair, we would expect 6 to come up 235/6 = 39.17 times. Is the proportion of 6s significantly higher than would be expected by chance, on the null hypothesis of a fair die?

The article then goes on to compute the p-value for a one-tailed binomial test (i.e. it finds the probability of getting 51 or more sixes from 235 rolls).

Then the article says that we might be interested in using a two-tailed p-value (Excerpt is quoted below). I have two questions:

  • What are the two tails we are looking at here (I think one is that $p(\#successes \geq 51)$. The other i guess would be of the form $p(\#successes \leq \bf{X})$, I am asking what is $\bf{X}$ in this example).
  • How did they calculate the probability of $.0437$. The explain it in the article -- I bolded this part -- but the explanation doesn't make sense to me.
    • (i.e. what is the "probability that the total deviation in numbers of events in either direction from the expected value"? I don't even get what the expected value is here? It would just be 39.17, the expected number of 6's in 235 rolls right?)

There are two methods to define the two tailed p-value. One method is to sum the probability that the total deviation in numbers of events in either direction from the expected value is either more than or less than the expected value. The probability of that occurring in our example is 0.0437. The second method involves computing the probability that the deviation from the expected value is as unlikely or more unlikely than the observed value, i.e. from a comparison of the probability density functions. This can create a subtle difference, but in this example yields the same probability of 0.0437

$\endgroup$
1
$\begingroup$

It's useful to consider a couple of things:

  1. the definition of a p value. It is the probability of obtaining a test statistic at least as extreme as the one we observed from the sample, if the null hypothesis were true.

  2. The particular test statistic we're using, which is a way of measuring how far the sample is from what we'd expect under the null; we choose it so that the particular alternative(s) we're interested in will tend to yield different test statistics than the null does.

    In particular, we generally choose the statistic so that it tends to either give large values or small values when the null hypothesis is false.

With those ideas in place, let's not discuss the calculations for the problem at hand.

Let's use $O_6$ to denote be the observed count of 6's in our experiment. In the case of this binomial proportion, we could look at measuring the difference of the observed count of 6's from the expected count (when the null is true), $T = O_6-\frac{235}{6}$ (or we could equivalently look at $T_2 = \frac{O_6}{235} - \frac16$, the difference of the proportion).

When looking at the one tailed alternative ($p_6>\frac16$), the test statistic would just be $T$, and values considerably larger than the expected difference of zero will be more consistent with the alternative hypothesis than they would be with the null.

Plot of binomial probabilities under H0 and the values greater than or equal to 51 marked in red

Note that what is plotted here is the count rather than the count minus its expected value; the appearance is the same, the only thing that is different is how the x-axis is labelled. The red parts represent the terms that make up the p value.

With a two tailed alternative, we could consider $T_3=|O_6-\frac{235}{6}|$ as our test statistic; this will make lower proportions than $\frac16$ and higher proportions than $\frac16$ both yield large values of the test statistic. Our measure of "more extreme" is now just the absolute difference of the observed count from that we'd expect if the null were true.

Plot of the absolute difference in count from expected and the values equal to or more than 51-39.17=11.83 marked in red

We could of course achieve the same end by considering our original $T$ but look at the distance from expected in both directions:

Plot of the (signed) difference in count from expected and the values at least as far from the means as 51-39.17=11.83 marked in red

$\endgroup$
  • $\begingroup$ First I just want to say that I think this is a really nice answer. Thank you. Secondly, a follow up question: If, for some reason, I wanted to do a one-tailed test but for $p<\frac16$ (when we have $O_6=51$), what should we change? Should we use a different test statistic (Maybe $T_3$)? Should we look at the left tail instead of the right? Intuitively, I think we should use a different test statistic, but couldn't we still use $T$ if we wanted? I think using $T_4=\frac{235}{6}-O_6$ and looking at the left tail would make sense. (I'm aware this alternative hypothesis doesn't make sense here) $\endgroup$ – user106860 Dec 17 '18 at 22:03
  • 1
    $\begingroup$ For a one-tailed test, you just stick with $T$ but you'd want to reject when it's unusually small (i.e. far into the negative range), rather than when it's unusually large. Your p-value will be all the probability (under H0) where $O_6\leq 51$ or $T\leq 51-39.17$ (i.e. the p-value will be well above 0.5, and you definitely won't reject, since the sample is saying that the population mean is not clearly smaller than 39.17; the sample mean is on the other side.) $\endgroup$ – Glen_b Dec 17 '18 at 23:58
1
$\begingroup$

Method One: Observed 51 is more than expected 39.17 by 11.83. If the observed number is less than expected 39.17 by 11.83 then observed number would be 27.34. So p value is calculated by $\Pr(X\ge 51) + \Pr(X\le 27)$.

Method Two: Get $\Pr(X=51)$. Calculate $\Pr(X=i), i = 0,..., 235$. Add all $\Pr(X=i)$ with $\Pr(X=i)\le \Pr(X=51)$. Of course there is easy way to get it.

$\endgroup$
  • $\begingroup$ Method 1 gives about $.035$ but the wikipedia article says the probability is $.0437$? (It is possible that I have a calculation error. Or maybe Wikipedia is using 28 instead of 27, but then that gives about $.045$, which might just be rounding error) $\endgroup$ – user106860 Dec 17 '18 at 22:11
  • 1
    $\begingroup$ I was struggling with 27 and 28, and finally though that 27 was correct one. It is possible that wiki used normal approximation. $\endgroup$ – user158565 Dec 17 '18 at 22:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.