# How few samples are needed to compare two means?

I have the following statistical problem:

I have two boxes, each has exactly 10000 numbers in it. Each number is either 0, 1, or 3. What I need to know is if the average amount in one box is bigger or smaller than the average amount in the other box.

Unfortunately, picking numbers is very costly for me. So what I would like to know is, how and how many numbers do I have to pick so that I can say with e.g. 90% probability that one box's average is below or above the other ones'?

EDIT:

I can iteratively randomly pick one number of a chosen box, and after each pick I can recalculate the probability. So I do not want to calculate how many I have to pick beforhand, but I would like to update a probability after each pick.

For example, when I randomly pick 10 numbers out of box 1 and each time I get a "3", and from the other box I pick 10 numbers and each one is "1", chances should be high that box 1's mean is higher than that of box 2. So I think it should be possible to somehow calculate if it is useful to continue picking or say with high probability that further picking will not change the final outcome.

This is no homework, but should be part of an optimization software I am writing. I want to quickly evaluate if one solution is better than another one.

• To give you some intuition, consider two scenarios. (1) Both boxes have 9999 1's; one has one 0 and another has one 3. (2) One box contains only 0's and the other contains only 3's. In scenario (1), you will have to pick all 10,000 numbers from each box before you can identify the one with the higher average; in (2), most routine tests will fail but the answer will be obvious after a half dozen draws from each box. Given such a range of possibilities, can you say anything more about what might be in your two boxes?
– whuber
Aug 10, 2011 at 19:34
• as @whuber points out, you need to know something about the posited 'effect size'. Here that would be defined as $\Delta =(\mu_1 - \mu_2) / \sigma$, where $\mu_i$ is the mean value in box $i$, and $\sigma$ is the pooled standard deviation. Given this, the sample size should be about $24 / \Delta^2$, with half of these drawn from each box. This should give you a 2-sided 2-sample t-test with 0.05 type I rate and 90 percent power assuming the effect size. Aug 10, 2011 at 20:18
• Is this a homework question? Aug 11, 2011 at 1:16
• @whuber This question is straight out of intro hypothesis testing. Is there a reason to address it as insoluble? Aug 11, 2011 at 1:22
• @Iterator: This may be a question straight out of stochastic process, namely, <a href="en.wikipedia.org/wiki/Two-armed_bandit">two-arm bandit problems</a> (and that's what the OP would want to look up -- this may be at least a partial answer). In my graduate level stochastic processes class, we did not cover it. Aug 11, 2011 at 6:20