# test if 5% difference is significant

I have a dataset that contains 20000 chess matches. According to these matches, white won 50% of the games while black won 45%. I want to test if this 5% difference is significant. How can I do that? I couldn't be sure which testing method to use.

The white won 10000 matches out of 20000. The black won 9000 matches out of 20000.

You can carry out a two-proportion z-test. For instance in R:

result <- prop.test(x = c(10000, 9000), n = c(20000, 20000))

Output:

2-sample test for equality of proportions with continuity correction

data:  c(10000, 9000) out of c(20000, 20000)
X-squared = 100.05, df = 1, p-value < 2.2e-16
alternative hypothesis: two.sided
95 percent confidence interval:
0.04017471 0.05982529
sample estimates:
prop 1 prop 2
0.50   0.45

The p-value is < 2.2e-16 so the difference is statistically significant.

(You can replace 20000 by 19000 if you only want to consider the matches won by either the white or black).

• Your test has rejected because 'data' $(10000, 9000)$ are not consistent with 50-50 white/black wins. You have not tested whether the actual hypothetical proportions are 50-45 were closely matched. (+1) anyhow, for suggesting prop.test. [The R procedure prop.test assumes equally likely outcomes as $H_0,$ unless different hypothetical values are specified as parameters of the procedure. ] Jan 31, 2021 at 23:01
• I suppose it depends how you interpret the OP's question. The purpose of the two-proportion z-test with equally likely outcomes as $H_0$ is to "determine whether the difference between two proportions is significant", which is what was requested. (+1) for the alternative interpretation. Jan 31, 2021 at 23:34

Suppose you have data for 10,000 matches. I use R to simulate results according to your 50% (1=white), 45% (2=black), 5% (3=draw) hypothesis.

set.seed(131)
x = sample(1:3, 10000, rep=T, p=(.50,.45,.05))
table(x)
x
1    2    3
5029 4499  472

In prop.test you can specify the hypothetical probabilities, as shown below.

prop.test(c(5029,4499,472), c(10000, 10000, 10000),
c(.5,.45,.05))

3-sample test for given proportions
without continuity correction

data:  c(5029, 4499, 472) out of c(10000, 10000, 10000),
null probabilities c(0.5, 0.45, 0.05)
X-squared = 1.9873, df = 3, p-value = 0.575
alternative hypothesis: two.sided
null values:
prop 1 prop 2 prop 3
0.50   0.45   0.05
sample estimates:
prop 1 prop 2 prop 3
0.5029 0.4499 0.0472

So the observed proportions were $$0.5029, 0.4499, 0.0472,$$ which are sufficiently close to the hypothetical proportions $$.50, .45, .05$$ to be considered consistent with the null hypothesis.

Over the long run, deviations (from hypothetical proportions) as large as seen in our simulation occur with probability about $$.58,$$ and so are not considered unusual.

Testing at the 5% level of significance (rejecting when the P-value is smaller the 0.05), we can expect this simulation experiment to give values inconsistent with the null hypothesis only 5% of the time.