2
$\begingroup$

Suppose one wants to test how many players still play the day after receiving the game, and two days after receiving it.

And that the results would be:

Game 1:

  • Total numbers of players = 120
  • Players playing the next day = 21,7%
  • Players playing two days after = 18,3%

Game 2:

  • Total numbers of players = 62
  • Players playing the next day = 16,1%
  • Players playing two days after = 8,1%

Is it possible to calculate how accurate the result is, particularly in game 2. What is the +- percentage?

Could game 2 in fact be better than game 1?

$\endgroup$
3
  • 1
    $\begingroup$ If players decide to play independently of each other, the answer is here. $\endgroup$
    – Scortchi
    Commented Jul 2, 2013 at 9:50
  • 1
    $\begingroup$ @Scortchi The question also appears to be asking (in the final sentence) about the standard error of the difference of two such percentages $\endgroup$
    – Glen_b
    Commented Jul 2, 2013 at 10:21
  • 1
    $\begingroup$ @Glen Good point: there's an answer for that here. $\endgroup$
    – Scortchi
    Commented Jul 2, 2013 at 12:24

1 Answer 1

1
$\begingroup$

Is it possible to calculate how accurate the result is, particularly in game 2. What is the +- percentage?

A common way to do this is to build a confidence interval (informally people look at this as a range of likely values even if this is not quite the right interpretation). There are many methods to do that, as detailed in the link provided by Scortchi (Confidence interval for Bernoulli sampling).

Could game 2 in fact be better than game 1?

Here, under the same assumptions (in particular that the players in each group are not the same and decide to continue to play independently of each other), you could use a $\chi^2$ test for independence. If the test allows you to reject the null hypothesis of independence, it means that the difference you observe is unlikely to have come about by chance. Since game 1 is better than game 2 in the sample, you would then conclude that it is indeed better than game 2 and, conversely, that game 2 is unlikely to be in fact better than game 1.

Even if the difference turns out not to be significant, it's still not evidence that there is no difference, let alone a difference in the other direction but that would at least mean that you cannot formally rule it out at the error level you picked. You could also compute a confidence interval for the difference (or for some other measure of the dependence like the odds ratio). Note that these methods (and others) can help you to gauge the strength of that evidence but no matter how you cut it, the data you have provide at least some evidence that game 1 is better and you can't wish that away.

Off the top of my head, I don't have any idea to use the data from both days together.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.