This may be a very well-known problem, but I'm not sure what methods would be good for this.

Let's assume that someone asks us to flip a coin 20 times to check whether it's a fair coin.

This is a fairly simple task. We could easily set up hypothesis tests and test p-value.

However, after this, let's assume that someone asks us to flip a coin 30 more times and see whether it's a fair coin.

My question is... how do we combine the previous data (that we gathered after we flipped the coin 20 times) with the new data (after we flip the coin 30 more times).

Is there a good way of doing so? Do we just combine both data (and assume that we just flipped the coin 50 times from the beginning) and check the hypothesis? Or is there a better way of doing this?

Any hint would be greatly appreciated.

  • $\begingroup$ One line of investigation would be to research "group sequential analysis." $\endgroup$ – whuber May 26 '15 at 20:16

If you didn't actually test the coin after the first set of tosses, or make the choice to toss the second set based on the first, then yes, you should be able to simply combine them into one large sample.

If your behavior was in any way contingent on what happened in the first set, then it affects the properties of the combined set when treated as one large sample, compared to what the properties would have been if it had actually been a large sample).

If you did test the coin after the first set, then the two tests will be dependent. (If you tested it but whatever the outcome it would have had no impact on anything in any way, you could continue to simply ignore it as if it had never happened. However, it may be better to report it as two tests, one of n=20 and a second of n=30 - there are various ways to combine independent tests after the fact ... but again, this would require that the second set not be contingent on the first.)

In response to an insightful question in comments:

It's not the proportion of heads that's at issue in my discussion (I can't bias the coin in this way). It's the test of it that's affected. If I assign significance to a particular set of outcomes, but I change the actions based on those outcomes, then I may change the proportion of times I'll make the various conclusions based on my experiment. whuber's comment on "best two out of three?" hints at this; it's not P(H) that is changed by saying "best two out of three" if tails comes up first, but the conclusion based on the experiment ("who wins the toss", in that case).

In the question's 20 tosses, say my original rule is "conclude the coin is biased toward heads if I see 15 or more heads, and conclude it's biased toward tails if I see 15 or more tails". About 2% of the time I'll call a fair coin biased toward heads and about 2% of the time I'll say it's biased toward tails.

Now consider the rule "conclude the coin is biased toward heads if I see 15 or more heads, otherwise, toss 30 more times and apply the rejection rule I would have applied as if I'd tossed 50 times to begin with", then I'm not longer as likely to say it's biased at all, nor as likely to say a fair coin is biased toward heads than to conclude it's biased toward tails

  • 1
    $\begingroup$ (+1) "I'll bet you--let's flip. I call heads. Rats, it was tails. Best two out of three? Best three out of five? ..." $\endgroup$ – whuber May 26 '15 at 13:34
  • 1
    $\begingroup$ How would combining both sets not improve the asymptotic tendency towards the law of large numbers, regardless of any a priori plans? What is the math behind the different scenarios? $\endgroup$ – Antoni Parellada May 26 '15 at 14:44
  • $\begingroup$ (Response has been moved to my answer.) $\endgroup$ – Glen_b May 26 '15 at 19:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.