# How to estimate the likelihood that a distribution is not random

Not sure I phrase the title right, but here's by question in a nutshell.

Say you have a box of balls with different colors. Let's say there are $N$ balls of $k$ different colors, and that the distribution of colors is such that some colors are more represented than others with a known distribution. You then ask a completely colorblind person to separate the balls across $M$ equally probable bins.

What is the likelihood of the distribution of balls made by this colorblind person?
Or: how likely is it that the person is really colorblind?

So far I've addressed the problem in this line of thought: If the person is completely colorblind, the most likely distribution would be random so it makes sense to somehow compare the outcome to this. If there were infinite balls I would just compare the distribution in each bin to the known distribution of colors, but as $N$ grows smaller this approach reduces in validity. The problem can also be recast such as to say that each bin draws a number of times from the distribution, but it gets very complicated when $N$ is finite because then the draws in each bin influence each other. Not sure how to proceed from here.

Disclaimer: This is a simplification of the actual problem I'm working on. Really, I'm studying whether the values a variable assumes in varying distances from a point can be said to have statistical significance, or are just random. The $N$ values of the variable are discretised into $k$ intervals, and distances into $M$ intervals.

As the problem is laid out it makes me think of a multivariate hypergeometric distribution. It is the same distribution assumed in Fisher's Exact Test.

That distribution models following situation :

In an urn there are $N$ balls each of a color so that there are $c$ colors. There are $K_i$ balls for color $i$. A sample of size $n$ is taken out of that urn and you end up with $k_i$ balls for each $i$ color so your sample is a vector ($k_1, k_2, ... k_c$).

The probability of such a sample is given by the multivariate hypergeometric distribution :

$P = \frac{\sum\limits_{i = 1}^c {K_i\choose k_i}}{{N \choose n}}$

In a Fisher's Exact Test (FET) for a table of size 2xc you can compute the probability of such a table happening with the multivariate hypergeometric distribution. Now, if you want to know how extreme that table is (i.e how not random it is) you have to consider all the other tables with same marginal totals (similar sampling) and compute their probability too. The p-value of that test is then the sum of all the probabilities lesser or equal to the probability of the actual sample/table.

Note that this distribution assume that there is no replacement and a ball is of only one color.

Hope it helps and I was clear

• Thanks for your answer! So in this setting I suppose you draw a sample vector for each bin. The only thing that's troubling here is that if N is finite, the probability distribution changes with each draw right? – Ulf Aslak Apr 20 '16 at 9:08
• Yeah I would say that each bin is a sample (so multi-multivariate ?). Yep, so I think that for each bin you have to adjust the $K_i$ to match with what is already out (to keep the assumption of no replacement). This way you would consider each bin as a MH sampling from a 'different' urn. – Riff Apr 20 '16 at 9:18
• Sure, but take into account that the samples are drawn in parallel and not necessarily with any particular governing order. – Ulf Aslak Apr 20 '16 at 9:23
• Hum... Then I guess there is no other way than modeling each sampling, one ball after the other to take that into account. At least none that I know of. But you already mentioned that one and it's going to get really heavy as $N$ grows. Even more so if you want to repeat that process multiple times to compare your situation to other, hypothetical, ones, in a FET-style of test. – Riff Apr 20 '16 at 9:28