I have a dataset that contains 20000 chess matches. According to these matches, white won 50% of the games while black won 45%. I want to test if this 5% difference is significant. How can I do that? I couldn't be sure which testing method to use.
2 Answers
The white won 10000 matches out of 20000. The black won 9000 matches out of 20000.
You can carry out a two-proportion z-test. For instance in R:
result <- prop.test(x = c(10000, 9000), n = c(20000, 20000))
Output:
2-sample test for equality of proportions with continuity correction
data: c(10000, 9000) out of c(20000, 20000)
X-squared = 100.05, df = 1, p-value < 2.2e-16
alternative hypothesis: two.sided
95 percent confidence interval:
0.04017471 0.05982529
sample estimates:
prop 1 prop 2
0.50 0.45
The p-value is < 2.2e-16 so the difference is statistically significant.
(You can replace 20000 by 19000 if you only want to consider the matches won by either the white or black).
-
$\begingroup$ Your test has rejected because 'data' $(10000, 9000)$ are not consistent with 50-50 white/black wins. You have not tested whether the actual hypothetical proportions are 50-45 were closely matched. (+1) anyhow, for suggesting
prop.test
. [The R procedureprop.test
assumes equally likely outcomes as $H_0,$ unless different hypothetical values are specified as parameters of the procedure. ] $\endgroup$– BruceETJan 31, 2021 at 23:01 -
1$\begingroup$ I suppose it depends how you interpret the OP's question. The purpose of the two-proportion z-test with equally likely outcomes as $H_0$ is to "determine whether the difference between two proportions is significant", which is what was requested. (+1) for the alternative interpretation. $\endgroup$– David M.Jan 31, 2021 at 23:34
Suppose you have data for 10,000 matches. I use R to simulate results according to your 50% (1=white), 45% (2=black), 5% (3=draw) hypothesis.
set.seed(131)
x = sample(1:3, 10000, rep=T, p=(.50,.45,.05))
table(x)
x
1 2 3
5029 4499 472
In prop.test
you can specify the hypothetical probabilities, as shown below.
prop.test(c(5029,4499,472), c(10000, 10000, 10000),
c(.5,.45,.05))
3-sample test for given proportions
without continuity correction
data: c(5029, 4499, 472) out of c(10000, 10000, 10000),
null probabilities c(0.5, 0.45, 0.05)
X-squared = 1.9873, df = 3, p-value = 0.575
alternative hypothesis: two.sided
null values:
prop 1 prop 2 prop 3
0.50 0.45 0.05
sample estimates:
prop 1 prop 2 prop 3
0.5029 0.4499 0.0472
So the observed proportions were $0.5029, 0.4499, 0.0472,$ which are sufficiently close to the hypothetical proportions $.50, .45, .05$ to be considered consistent with the null hypothesis.
Over the long run, deviations (from hypothetical proportions) as large as seen in our simulation occur with probability about $.58,$ and so are not considered unusual.
Testing at the 5% level of significance (rejecting when the P-value is smaller the 0.05), we can expect this simulation experiment to give values inconsistent with the null hypothesis only 5% of the time.