I am trying to solve the following question:
Player A won 17 out of 25 games while player B won 8 out of 20 - is there a significant difference between both ratios?
The thing to do in R that comes to mind is the following:
> prop.test(c(17,8),c(25,20),correct=FALSE) 2-sample test for equality of proportions without continuity correction data: c(17, 8) out of c(25, 20) X-squared = 3.528, df = 1, p-value = 0.06034 alternative hypothesis: two.sided 95 percent confidence interval: -0.002016956 0.562016956 sample estimates: prop 1 prop 2 0.68 0.40
So this test says that the difference is not significant at the 95% confidence level.
Because we know that
prop.test() is only using an approximation I want to make things more exact by using an exact binomial test - and I do it both ways around:
> binom.test(x=17,n=25,p=8/20) Exact binomial test data: 17 and 25 number of successes = 17, number of trials = 25, p-value = 0.006693 alternative hypothesis: true probability of success is not equal to 0.4 95 percent confidence interval: 0.4649993 0.8505046 sample estimates: probability of success 0.68 > binom.test(x=8,n=20,p=17/25) Exact binomial test data: 8 and 20 number of successes = 8, number of trials = 20, p-value = 0.01377 alternative hypothesis: true probability of success is not equal to 0.68 95 percent confidence interval: 0.1911901 0.6394574 sample estimates: probability of success 0.4
Now this is strange, isn't it? The p-values are totally different each time! In both cases now the results are (highly) significant but the p-values seem to jump around rather haphazardly.
- Why are the p-values that different each time?
- How to perform an exact two sample proportions binomial test in R correctly?