1
$\begingroup$

I have data from 1000 games, all performed on a group of 4 players (the group and the players are constant). For each game, I have 5 possible results -> player A wins, player B wins, player C wins, player D wins and a draw. I can of course sum up the results and if I get sums like "A - 263, B - 233, C - 246, D - 237, 21 draws" I can guess that it probably means these players have no difference of skills, or the game is random. But how would I go about proving that with statistics?

Ultimately I would like to be able to use the ratio of the wins to confirm if one of the players is significantly "better" than the others - has an advantage that goes beyond randomness. I've taken some statistics, including learning about Z-tests, t-tests, ANOVA and so forth but I'm having trouble matching this real situation to the theory I have read about.

$\endgroup$

1 Answer 1

2
$\begingroup$

There are a few different approach you could take with this. If you are simply trying to find statistical evidence that the players do not all have the same probability of winning (i.e. some players may win more than others), you could start with a chi-squared test for goodness of fit. This test attempts to determine if a discrepancy exists between each player's observed win frequency and those expected under the null hypothesis, $H_0$. In this particular case, you'd assume that each player has an equal chance of winning a game. Since there are 5 players that implies each player has a 1 in 5 chance of winning a game. So Under the Null Hypothesis you'd have:

\begin{eqnarray*} H_{0}: & p_{1}= & p_{2}=p_{3}=p_{4}=p_{5}=1/5 \end{eqnarray*}

Where player $i$'s probability of winning is denoted by $p_i$ for $i=1, 2, 3, 4, 5$ for each of the $k=5$ players.

Then a useful measure for the overall discrepancy between the observed and expected player win frequencies is given by the chi-squared statistic: \begin{eqnarray*} \chi^{2} & =\sum\frac{(Observed-Expected)^2}{Expected} & \frac{\sum_{i=1}^{k}(n_{i}-np_{i0})^{2}}{np_{i0}} \end{eqnarray*}

Where $O$ and $E$ are the observed an expected frequencies respectively, $n_i$ is the number of games won by player $i$, and $p_i=1/5$ is proportion of the games you'd expect each player to win under the assumption that all players are equally skilled at winning the game. This statistic is distributed according to the chi-squared distribution with degrees of freedom equal to $k-1$ (1 fewer than the number of players). You would then reject the null hypothesis when the computed Chis-squared test statistic is greater than or equal to the value of the chi-squared distribution at the selected level of $\alpha$ for your test (we'll use $\alpha=0.05$). In other words,

\begin{eqnarray*} \text{Reject}\,H_{0}\,\text{if } & \chi^{2}\ge & \chi_{\alpha}^{2} \end{eqnarray*}

So we can carry out this test by hand since it's simple enough:

\begin{eqnarray*} \chi^{2} & = & \frac{(263-200)^{2}+(233-200)^{2}+(246-200)^{2}+(237-200)^{2}+(21-200)^{2}}{200}\\ & = & 202.92 \end{eqnarray*}

Comparing this to a chi-squared distribution table with $k-1=4$ degrees of freedom, we see that, under the null hypothesis we'd expect a $\chi^2$ of less than 9.49 if we are to believe our null hypothsis. However, since $\chi^2_{0.05} \ge 9.49$, we find strong evidence to reject the null hypothesis and conclude there is evidence with this data to believe that players do not have an equal chance of winning the game.

This test can easily be performed in R too:

#Assign number of wins to each player
n_i<-c(263, 233, 246, 237, 21)
#Assign equal probabilities of winning to each player
p_i=rep(1/5, 5)
#Perform the test
chisq.test(x = n_i, p = p_i)

Which produces the following output:

    Chi-squared test for given probabilities

data:  n_i
X-squared = 202.92, df = 4, p-value < 2.2e-16

Since the $p$-value associated with this test is very small, we reach the same conclusion as we did with our previous hand calculation.

Of course there are other tests you could do as well like a logistic regression and that test might be better suited if you wanted to control on other variables like the amount of time each player has spent playing the game, gender, age, etc. But this is a good test to use if no other information is available for which you'd like to control.

Best of luck to you.

$\endgroup$
3
  • 1
    $\begingroup$ Nice to show both hand computation and results from R (+1) $\endgroup$
    – BruceET
    Aug 25, 2020 at 21:50
  • $\begingroup$ Thank you so much! It makes a lot of sense now! I've also did some more reading and it would also be possible to test it with a binomial test (with probability 25% assuming we have 4 players) for a specific player, assuming player A wins = k, and all the other player wins = player A losses, right? This way an alternative hypothesis could straightaway be "this player's probability to win is higher than the random 25%". $\endgroup$
    – bst
    Aug 25, 2020 at 21:51
  • $\begingroup$ With a binomial test, you'd be testing only one player. You wouldn't want to do the binomial test 4 times (once for each player) as in this case every time you perform the test, you inflate the Type I error (you increase you chances of finding a non-significant results significant by chance alone). $\endgroup$ Aug 25, 2020 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.