When $p = .6$, then the binomial distribution is not symmetric. In this situation, how a "two-sided p-value" when observing 2 successes in 15 trials is calculated?

In R, such a p-value is obtained as follows:

binom.test(2, 15, p = .6, "two.sided")$p.value ## How this p-value has been calculated

Therefore, I'm asking what is the statistical basis for the calculation of 2-sided p-value in the above case?

A picture for my question:

enter image description here

  • $\begingroup$ Hint: what are the mean and variance of this distribution? $\endgroup$ – TheIncredibleZ1 Jun 10 '17 at 15:40
  • $\begingroup$ I would attempt to approximate this by an appropriate normal random variable. $\endgroup$ – TheIncredibleZ1 Jun 10 '17 at 16:04
  • 1
    $\begingroup$ This is strongly related to stats.stackexchange.com/questions/140107. Your Q might even be considered a duplicate of that one. $\endgroup$ – amoeba May 8 '18 at 12:39

Exact p-value

The meaning of the p-value is the probability of getting the sample or more extreme than sample under the hull hypothesis. In your question, the distribution under null hypothesis is Binomial with probability 0.6 on 15 trials. As you did on that graph, you already got the probability of 0, 1, 2,..., 15 successes among 15 trials. In your sample, you observed 2 successes. Next, comparing $\Pr(Y=x), x =0,1,2,...,15$ with $\Pr(Y=2)$. If $\Pr(Y=x) <= \Pr(Y=2)$, we consider $Y=x$ is extreme situation, so add $\Pr(Y=x)$ into your p-value. After go through all of the x, you get you exact p-value for 2-side test.

| cite | improve this answer | |
  • $\begingroup$ Sorry I am not familiar with R. I think you already finished the calculation based on the graph you presented. $\endgroup$ – user158565 Jun 10 '17 at 18:56
  • $\begingroup$ you can go to stattrek.com/online-calculator/binomial.aspx. First get $\Pr(Y=2) = 0.00025367150592$. Only Y=0,1 are the extreme situations, so p-value should be $\Pr(Y<=2) = 0.000278904438784$. $\endgroup$ – user158565 Jun 10 '17 at 19:04
  • 1
    $\begingroup$ If you check probability for $Y=0, 1,...,15$, you will find that $\Pr(Y=0) <= \Pr(Y=2)$ and $\Pr(Y=1) < \Pr(Y=2)$. For $Y=3,..,15$, their prob. are > $\Pr(Y=2)$. So the 2-side p-value is $\Pr(Y=0) +\Pr(Y=1) +\Pr(Y=2) = \Pr(Y<=2) = 0.0002789044$ $\endgroup$ – user158565 Jun 10 '17 at 19:54
  • 1
    $\begingroup$ If you observed Y=3, then Y = 0,1,2,3 and 15 are extreme situation, because their prob. are <= $\Pr(Y=3)= 0.00164886478848$, so the two side p value is $\Pr(Y<=3) +\Pr(Y=15)$. Need to find extreme situations when you observed situation is changed (Y=2 changes to Y=3) $\endgroup$ – user158565 Jun 10 '17 at 20:58
  • $\begingroup$ Yes, you are correct. When Y = 2, there is no extreme on large Y side, so just count smaller Y side. Maybe you try Y =4 to see if you can get correct answer. $\endgroup$ – user158565 Jun 10 '17 at 21:20

Here is how I interpret your question: suppose a trial indicates a success probability of 60%, and we record 2 successes out of 15. We determine if this claim is valid.

Let $X$ be a random variable denoting these successes. Then $\bar{x} = 9$ and $\sigma = \frac{6}{\sqrt{10}}$. If we naively approximate a normal random variable for this data, we obtain a z-score of approximately $-3.689$ to which we obtain a p-value of $0.0002249$. We can reject our null hypothesis that the success probability is 60%.

If this is not the intent of the question, please edit the original post to include as much information as possible.

| cite | improve this answer | |
  • $\begingroup$ The same way the mean and variance are calculated for the binomial distribution. $\endgroup$ – TheIncredibleZ1 Jun 10 '17 at 17:12
  • $\begingroup$ Note that $\sqrt{3.6} = \sqrt{\frac{36}{10}} = \frac{6}{\sqrt{10}}$. $\endgroup$ – TheIncredibleZ1 Jun 10 '17 at 17:30
  • $\begingroup$ R, Excel, your TI calculator, they all do this the same way. Read the associated documentation and your class notes and text for more info. (not professional, just a maths/industrial engineering student) $\endgroup$ – TheIncredibleZ1 Jun 10 '17 at 17:36

You can get a first approximation to the Clopper-Pearson interval by assuming the distribution is symmetric, in which case the events Y <= 2 are "just as extreme" as the events Y >= 15-2 = 13, and your approximate p-value is

p-value_1 = Pr(Y <= 2) + Pr( Y >= 13)

A glance at your picture reveals that symmetry wasn't a good assumption, as the event Y = 13 is quite a bit more likely than the event Y = 2. Indeed the event Y=14 is also more likely than the event Y=2. But the event Y=15 seems (from your picture) to be less likely than the event Y=2, so you might include it in your second-approximation to the p-value calculation (which should sum the probability of your observed event and of all "more extreme" events):

p-value_2 = Pr(Y <= 2) + Pr( Y >= 15)

For your third (and final) approximation, you'd check the probability density at Y=15. I'm confident a_statistician got it right, with Pr(Y=15) < Pr(Y=2), so your two-tailed p-value is

p-value = Pr(Y <= 2) + Pr( Y >= 16) = Pr(Y <= 2)

As pointed out in Two-sided binomial test in Excel, the Clopper-Pearson 2-sided binomial test isn't something you'd want to perform in Excel. You can hack your way through it for particular cases such as the one in your diagram. Someday, someone somewhere will go to the trouble of developing and carefully-validating and publishing a Visual Basic routine BINOM.TEST.TWOSIDED. But... why not learn S, or pay the licensing fees for SPSS?

| cite | improve this answer | |

This should get close.

Edit* This code below is Visual Basic for Applications which you would run in Developer mode in Excel. It implements the solution outlined by a-statistician above. First it calculates the probability (oP) of Hits out of trials with P probability. Then, to find the extremes at each tail it cycles through all possible hits in trials and when it finds a probability (nP) that is more extreme (i.e., less than) the oP value in question, it adds it to the probability.

Function binom_test_tt(hits As Integer, trials As Integer, P As Double)

Dim i As Integer
Dim oP As Double
Dim nP As Double
Dim sum As Double

oP = Application.WorksheetFunction.Binom_Dist(hits, trials, P, False)

sum = oP

i = trials

While i <> -1  'cycle through all possible values

' get a new probability
nP = Application.WorksheetFunction.Binom_Dist(i, trials, P, False) 

'if more extreme than op, add it into the sum
 If (nP < oP) Then sum = sum + _ 
            Application.WorksheetFunction.Binom_Dist(i, trials, P, False) 

i = i - 1


binom_test_tt = sum

End Function
| cite | improve this answer | |
  • 1
    $\begingroup$ Although implementation is often mixed with substantive content in questions, we are supposed to be a site for providing information about statistics, machine learning, etc., not code. It can be good to provide code as well, but please elaborate your substantive answer in text for people who don't read this language well enough to recognize & extract the answer from the code. $\endgroup$ – gung - Reinstate Monica May 8 '18 at 12:25
  • $\begingroup$ Thanks for the heads-up gung. I have added an explanation to indicate how the code implements the explanation provided by a-statistician above. $\endgroup$ – DrJBN May 9 '18 at 14:03

@a_statistician correctly points out to the fact that p-value refers to the probability of observed or extreme outcomes, given our null hypothesis is true.

Look, here our null hypotheses is [Ho : p = 0.6]

And a two sided hypothesis test implies that our alternative hypothesis is [Ha: p != 0.1]

So as per the definition of p-value, our answer is simply the mathematical equation: we have our observed outcome to be p = 2.

p-value = P[k <= 2] + P[k >= 2], where k is my no. of successes. Apply the usual formula and you'll get at your answer.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.