I am trying to solve the following question:
Player A won 17 out of 25 games while player B won 8 out of 20 - is there a significant difference between both ratios?
The thing to do in R that comes to mind is the following:
> prop.test(c(17,8),c(25,20),correct=FALSE)
2-sample test for equality of proportions without continuity correction
data: c(17, 8) out of c(25, 20)
X-squared = 3.528, df = 1, p-value = 0.06034
alternative hypothesis: two.sided
95 percent confidence interval:
-0.002016956 0.562016956
sample estimates:
prop 1 prop 2
0.68 0.40
So this test says that the difference is not significant at the 95% confidence level.
Because we know that prop.test()
is only using an approximation I want to make things more exact by using an exact binomial test - and I do it both ways around:
> binom.test(x=17,n=25,p=8/20)
Exact binomial test
data: 17 and 25
number of successes = 17, number of trials = 25, p-value = 0.006693
alternative hypothesis: true probability of success is not equal to 0.4
95 percent confidence interval:
0.4649993 0.8505046
sample estimates:
probability of success
0.68
> binom.test(x=8,n=20,p=17/25)
Exact binomial test
data: 8 and 20
number of successes = 8, number of trials = 20, p-value = 0.01377
alternative hypothesis: true probability of success is not equal to 0.68
95 percent confidence interval:
0.1911901 0.6394574
sample estimates:
probability of success
0.4
Now this is strange, isn't it? The p-values are totally different each time! In both cases now the results are (highly) significant but the p-values seem to jump around rather haphazardly.
My questions
- Why are the p-values that different each time?
- How to perform an exact two sample proportions binomial test in R correctly?
prop.test
vschisq.test
), the same underlying concept is in this question. You are running three different tests with different "null hypothesis" in each of your three examples. $\endgroup$