I have a problem that seems tractable, which I will attempt to describe with classic balls and buckets probability nomenclature.

Imagine someone brings you a bag of colored balls. This sample was selected at random from one of many large buckets of balls. Each bucket contains 1,000,000 balls, and each ball is one of 1,000 colors. For any bucket, the proportion of balls for each color is fixed, and we know what it is. For example, Bucket A always has 0.02 Red, 0.013 Green, 0.009 Blue, etc.

So--how would I go about estimating which bucket the sample of balls came from? I know that if I have a bag (sample) of 1,000,000 balls, I can say unambiguously which bucket that came from, as the proportions are defined. However, what do I do if I have only 2000 balls in the sample? It seems to me I should be able to calculate quite specifically the likelihood that the sample came from a particular bucket.

Any help would be greatly appreciated! I'll be doing this in code (likely Python).

Best regards, Dan

  • 1
    $\begingroup$ Are you asking how to compute the likelihood or are you asking whether it's sensible to select the bucket with the highest likelihood? $\endgroup$
    – ziggystar
    Mar 14, 2013 at 19:44
  • $\begingroup$ Good question--I am asking both to some degree, I suppose. It also seems to me that they would be related questions--for example, below some threshold of 'sensible' fit, we would conclude that we don't know which bucket the sample came from. $\endgroup$ Mar 15, 2013 at 0:29

1 Answer 1


Under the assumption that you are drawing with replacement (which is the simpler case), you are simply dealing with a sample from a multinomial distribution and the urn defines the parameters. This assumption may be reasonable depending on the exact numbers, i.e. whether the removal of balls in the size of your sample significantly alters the proportions between the colors in a certain bucket. So when $X_i$ is the number of balls of color $i$, and $p_i$ is the probability to draw a ball of color $i$ from the selected bucket, and you draw a total of $n$ balls. Then the probability (or likelihood) of drawing a particular sample from a particular bucket is given by the following formula. $$ \Pr(X_1 = x_1,\dots,X_k = x_k) = {n! \over x_1!\cdots x_k!}p_1^{x_1}\cdots p_k^{x_k} $$

Then you can use Bayes Theorem to calculate the probability distribution over buckets given a sample $\Pr(B=b|X=x)$:

$$ \Pr(B=b|X=x) = \frac{\Pr(X=x|B=b)\Pr(B=b)}{\Pr(X=x)} \propto \Pr(X=x|B=b)P(B=b) $$

Under the assumption, that every bucket has equal probability to get selected, you arrive at $\Pr(B = b|X = x) \propto \Pr(X=x|B=b)$. You obtain this distribution, simply by by calculating the likelihoods for every bucket and a normalization of those.

Without replacement

When you want to model your problem using drawing without replacement, you're dealing with a multivariate hypergeometric distribution, instead of the multinomial distribution and have to use the according formula.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.