I have the following statistical problem:
I have two boxes, each has exactly 10000 numbers in it. Each number is either 0, 1, or 3. What I need to know is if the average amount in one box is bigger or smaller than the average amount in the other box.
Unfortunately, picking numbers is very costly for me. So what I would like to know is, how and how many numbers do I have to pick so that I can say with e.g. 90% probability that one box's average is below or above the other ones'?
I can iteratively randomly pick one number of a chosen box, and after each pick I can recalculate the probability. So I do not want to calculate how many I have to pick beforhand, but I would like to update a probability after each pick.
For example, when I randomly pick 10 numbers out of box 1 and each time I get a "3", and from the other box I pick 10 numbers and each one is "1", chances should be high that box 1's mean is higher than that of box 2. So I think it should be possible to somehow calculate if it is useful to continue picking or say with high probability that further picking will not change the final outcome.
This is no homework, but should be part of an optimization software I am writing. I want to quickly evaluate if one solution is better than another one.