4
$\begingroup$

Two friends and I have been playing a game that is a combination of skill and luck. (as most games are). We assume that if the game was all luck and/or we all had the same skill level eventually the win% of all of us would approach 33%.

So what sample size would we need to show that one of us was "better" at the game? That is if we played 50 games, what % of games (or how far from 33%) would someone need to demonstrate skill vs. luck (perhaps at a 95% or 99% likelihood). What if we played 100 games or 200 games?

I would assume we would need a smaller percentage difference from 33% as the sample size went up.

$\endgroup$
5
$\begingroup$

There is a fundamental problem with analyzing multiplayer games. If 2 players can gang up on a third, then "equal skill" does not guarantee equal average results.

A more subtle phenomenon is that a player can become the "king-maker." The king-maker can't win, but can decide which opponent wins. If that player doesn't like you, the king-maker can decide that you don't win.

So, you need to add more assumptions. A reasonable model is that you are sampling independently from a distribution on a $3$-element set, and you would like to distinguish the distribution from $( \frac{1}{3},\frac{1}{3},\frac{1}{3})$. The usual way to do this is with a multinomial test. You can also use simpler but less powerful binomial tests to test each frequency against $\frac{1}{3}$.

In practice I would use a normal approximation to the binomial distribution with continuity correction, and a threshold tighter than $0.05$ to reflect that any of the $3$ players might be strongest. Suppose the probability of winning is $\frac13$. The standard deviation after $n$ games is $\sqrt{n (\frac13)(1-\frac13)}$. A result of $40$ in $100$ games would be $6.67$ games above the mean, but $40$ should be viewed as the interval $[39.5,40.5]$, so it should be viewed as $6.17$ above the mean under the continuity correction. This is $6.17\bigg/(10\sqrt{\frac29}) = 1.3$ standard deviations above the mean, which wouldn't be strong evidence that you are stronger than average. At $43 (1.94\sigma), 44 (2.16\sigma),$ or $45 (2.37\sigma)$ you might conclude that you have strong evidence that you are stronger than average.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.