5
$\begingroup$

When $p = .6$, then the binomial distribution is not symmetric. In this situation, how a "two-sided p-value" when observing 2 successes in 15 trials is calculated?

In R, such a p-value is obtained as follows:

binom.test(2, 15, p = .6, "two.sided")$p.value ## How this p-value has been calculated

Therefore, I'm asking what is the statistical basis for the calculation of 2-sided p-value in the above case?

A picture for my question:

enter image description here

Improve this question
$\endgroup$
3
  • $\begingroup$ Hint: what are the mean and variance of this distribution? $\endgroup$
    – Sean Roberson
    Jun 10, 2017 at 15:40
  • $\begingroup$ I would attempt to approximate this by an appropriate normal random variable. $\endgroup$
    – Sean Roberson
    Jun 10, 2017 at 16:04
  • 2
    $\begingroup$ This is strongly related to stats.stackexchange.com/questions/140107. Your Q might even be considered a duplicate of that one. $\endgroup$
    – amoeba
    May 8, 2018 at 12:39

6 Answers 6

Reset to default
7
$\begingroup$

Exact p-value

The meaning of the p-value is the probability of getting the sample or more extreme than the sample under the null hypothesis. In your question, the distribution under the null hypothesis is Binomial with probability 0.6 on 15 trials. As you did on that graph, you already got the probability of 0, 1, 2,..., 15 successes among 15 trials. In your sample, you observed 2 successes. Next, comparing $\Pr(Y=x), x =0,1,2,...,15$ with $\Pr(Y=2)$. If $\Pr(Y=x) <= \Pr(Y=2)$, we consider $Y=x$ is extreme situation, so add $\Pr(Y=x)$ into your p-value. After going through all of the x, you get your exact p-value for the 2-side test.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Sorry I am not familiar with R. I think you already finished the calculation based on the graph you presented. $\endgroup$
    – user158565
    Jun 10, 2017 at 18:56
  • $\begingroup$ you can go to stattrek.com/online-calculator/binomial.aspx. First get $\Pr(Y=2) = 0.00025367150592$. Only Y=0,1 are the extreme situations, so p-value should be $\Pr(Y<=2) = 0.000278904438784$. $\endgroup$
    – user158565
    Jun 10, 2017 at 19:04
  • 1
    $\begingroup$ If you check probability for $Y=0, 1,...,15$, you will find that $\Pr(Y=0) <= \Pr(Y=2)$ and $\Pr(Y=1) < \Pr(Y=2)$. For $Y=3,..,15$, their prob. are > $\Pr(Y=2)$. So the 2-side p-value is $\Pr(Y=0) +\Pr(Y=1) +\Pr(Y=2) = \Pr(Y<=2) = 0.0002789044$ $\endgroup$
    – user158565
    Jun 10, 2017 at 19:54
  • 2
    $\begingroup$ If you observed Y=3, then Y = 0,1,2,3 and 15 are extreme situation, because their prob. are <= $\Pr(Y=3)= 0.00164886478848$, so the two side p value is $\Pr(Y<=3) +\Pr(Y=15)$. Need to find extreme situations when you observed situation is changed (Y=2 changes to Y=3) $\endgroup$
    – user158565
    Jun 10, 2017 at 20:58
  • 1
    $\begingroup$ Yes, you are correct. When Y = 2, there is no extreme on large Y side, so just count smaller Y side. Maybe you try Y =4 to see if you can get correct answer. $\endgroup$
    – user158565
    Jun 10, 2017 at 21:20
0
$\begingroup$

Here is how I interpret your question: suppose a trial indicates a success probability of 60%, and we record 2 successes out of 15. We determine if this claim is valid.

Let $X$ be a random variable denoting these successes. Then $\bar{x} = 9$ and $\sigma = \frac{6}{\sqrt{10}}$. If we naively approximate a normal random variable for this data, we obtain a z-score of approximately $-3.689$ to which we obtain a p-value of $0.0002249$. We can reject our null hypothesis that the success probability is 60%.

If this is not the intent of the question, please edit the original post to include as much information as possible.

Improve this answer
$\endgroup$
3
  • $\begingroup$ The same way the mean and variance are calculated for the binomial distribution. $\endgroup$
    – Sean Roberson
    Jun 10, 2017 at 17:12
  • $\begingroup$ Note that $\sqrt{3.6} = \sqrt{\frac{36}{10}} = \frac{6}{\sqrt{10}}$. $\endgroup$
    – Sean Roberson
    Jun 10, 2017 at 17:30
  • $\begingroup$ R, Excel, your TI calculator, they all do this the same way. Read the associated documentation and your class notes and text for more info. (not professional, just a maths/industrial engineering student) $\endgroup$
    – Sean Roberson
    Jun 10, 2017 at 17:36
0
$\begingroup$

You can get a first approximation to the Clopper-Pearson interval by assuming the distribution is symmetric, in which case the events Y <= 2 are "just as extreme" as the events Y >= 15-2 = 13, and your approximate p-value is

p-value_1 = Pr(Y <= 2) + Pr( Y >= 13)

A glance at your picture reveals that symmetry wasn't a good assumption, as the event Y = 13 is quite a bit more likely than the event Y = 2. Indeed the event Y=14 is also more likely than the event Y=2. But the event Y=15 seems (from your picture) to be less likely than the event Y=2, so you might include it in your second-approximation to the p-value calculation (which should sum the probability of your observed event and of all "more extreme" events):

p-value_2 = Pr(Y <= 2) + Pr( Y >= 15)

For your third (and final) approximation, you'd check the probability density at Y=15. I'm confident a_statistician got it right, with Pr(Y=15) < Pr(Y=2), so your two-tailed p-value is

p-value = Pr(Y <= 2) + Pr( Y >= 16) = Pr(Y <= 2)

As pointed out in Two-sided binomial test in Excel, the Clopper-Pearson 2-sided binomial test isn't something you'd want to perform in Excel. You can hack your way through it for particular cases such as the one in your diagram. Someday, someone somewhere will go to the trouble of developing and carefully-validating and publishing a Visual Basic routine BINOM.TEST.TWOSIDED. But... why not learn S, or pay the licensing fees for SPSS?

Improve this answer
$\endgroup$
0
$\begingroup$

This should get close.

Edit* This code below is Visual Basic for Applications which you would run in Developer mode in Excel. It implements the solution outlined by a-statistician above. First it calculates the probability (oP) of Hits out of trials with P probability. Then, to find the extremes at each tail it cycles through all possible hits in trials and when it finds a probability (nP) that is more extreme (i.e., less than) the oP value in question, it adds it to the probability.

Function binom_test_tt(hits As Integer, trials As Integer, P As Double)

Dim i As Integer
Dim oP As Double
Dim nP As Double
Dim sum As Double

oP = Application.WorksheetFunction.Binom_Dist(hits, trials, P, False)

sum = oP

i = trials


While i <> -1  'cycle through all possible values

' get a new probability
nP = Application.WorksheetFunction.Binom_Dist(i, trials, P, False) 

'if more extreme than op, add it into the sum
 If (nP < oP) Then sum = sum + _ 
            Application.WorksheetFunction.Binom_Dist(i, trials, P, False) 

i = i - 1

Wend


binom_test_tt = sum


End Function
Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Although implementation is often mixed with substantive content in questions, we are supposed to be a site for providing information about statistics, machine learning, etc., not code. It can be good to provide code as well, but please elaborate your substantive answer in text for people who don't read this language well enough to recognize & extract the answer from the code. $\endgroup$
    – gung - Reinstate Monica
    May 8, 2018 at 12:25
  • $\begingroup$ Thanks for the heads-up gung. I have added an explanation to indicate how the code implements the explanation provided by a-statistician above. $\endgroup$
    – DrJBN
    May 9, 2018 at 14:03
0
$\begingroup$

With a two-sided test, you should always look at both tails of the distribution. It just so happens in this example that looking at one tail gives the correct p-value. This is because the observed value of 2 is really small.

More generally, though, you should add all probabilities precisely for those values of Y with probabilities (under the null hypothesis) not exceeding the probability that Y = 2. The following R code provides this calculation:

null_distribution <- dbinom(x = 0:15, size = 15, prob = 0.6)

prob_of_observed_value <- dbinom(x = 2, size = 15, prob = 0.6)

(index_with_equal_or_less_prob <- which(null_distribution <= prob_of_observed_value))
#> [1] 1 2 3

# these are the probabilities corresponding to Y=0,1,2
dbinom(x = 0:15, size = 15, p = 0.6)[index_with_equal_or_less_prob]
#> [1] 1.073742e-06 2.415919e-05 2.536715e-04

sum(dbinom(x = 0:15, size = 15, p = 0.6)[index_with_equal_or_less_prob])
#> [1] 0.0002789044

Created on 2021-11-02 by the reprex package (v0.3.0)

If the observed value was 3, we immediately see that we must also include $P(Y = 15)$:

null_distribution <- dbinom(x = 0:15, size = 15, prob = 0.6)

prob_of_observed_value <- dbinom(x = 3, size = 15, prob = 0.6)

(index_with_equal_or_less_prob <- which(null_distribution <= prob_of_observed_value))
#> [1]  1  2  3  4 16

dbinom(x = 0:15, size = 15, p = 0.6)[index_with_equal_or_less_prob]
#> [1] 1.073742e-06 2.415919e-05 2.536715e-04 1.648865e-03 4.701850e-04

sum(dbinom(x = 0:15, size = 15, p = 0.6)[index_with_equal_or_less_prob])
#> [1] 0.002397954

binom.test(3, 15, 0.6)
#> 
#>  Exact binomial test
#> 
#> data:  3 and 15
#> number of successes = 3, number of trials = 15, p-value = 0.002398
#> alternative hypothesis: true probability of success is not equal to 0.6
#> 95 percent confidence interval:
#>  0.04331201 0.48089113
#> sample estimates:
#> probability of success 
#>                    0.2

Created on 2021-11-02 by the reprex package (v0.3.0)

Improve this answer
$\endgroup$
-1
$\begingroup$

@a_statistician correctly points out to the fact that p-value refers to the probability of observed or extreme outcomes, given our null hypothesis is true.

Look, here our null hypotheses is [Ho : p = 0.6]

And a two sided hypothesis test implies that our alternative hypothesis is [Ha: p != 0.1]

So as per the definition of p-value, our answer is simply the mathematical equation: we have our observed outcome to be p = 2.

p-value = P[k <= 2] + P[k >= 2], where k is my no. of successes. Apply the usual formula and you'll get at your answer.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.