# How different random number generators can be more similar than identical ones?

I have 6 random number generators. They are "black boxes", i.e. I do not know if they are the same or different. For example I do not know if they provide the same arithmetic averages and/or root mean square deviations. My goal is to check if they are the same or different. The problem is that I can generate a limited set of numbers (every random number generator gives me only 20 numbers).

The procedure that I use is as following.

1. I choose a pair of generators.
2. I use each of them to generate 20 numbers.
3. For every generator I use these numbers to calculate the mean.
4. Then I calculate the difference between the means (mean1 - mean2). I will call this difference as "reference difference".

Now I want to know if this difference is "real". In other words I want to know if the observed difference in means is caused by the fact that I do have two different random number generators or I have identical generators and the mean is different just because of chance (small number of numbers).

To answer the last question I use the following procedure.

1. I combine the two sets of numbers (20 numbers + 20 numbers).
2. Then I randomly split this set into two subsets of equal size.
3. For each subset I calculate means and, as before, I calculate the difference between the two means (I will call this difference as "test difference").
4. I compare this test difference with the reference difference.
5. I repeat this procedure many times and to see in home many cases the test difference is smaller then the reference difference.

I thought that if the reference difference is larger than the test difference in most of the cases than it is conditioned not by chance but by the fact that the compared random number generators are different. In contrast, if the generators are identical than the test difference has 50% of chances to be larger (or smaller) then the reference one.

However, I see that for the most of the considered pairs of the generators the reference difference between the averages is smaller than the test differences. How could it be? It looks like the the generators are more similar than identical generators?

• The widely used random number generators today are of high quality that will not allow to distinguish between them using small samples. Apr 28 '11 at 10:55
• Do you really mean "random number generators", i.e. numerical algorithms that aim to give observations with a known distribution (usually uniform(0,1)), or just "random variables", i.e. sources of data with unknown distributions. Apr 28 '11 at 16:11
• Apr 28 '11 at 22:32
• @Aniko, you are right, it is better to say that I have a random variable. I just wanted to abstract from the details. In my case the real numbers are created by different person (one "random number generator" in my case is a human). Apr 29 '11 at 7:59
• When you say "reference differences" are smaller than the "test differences", do you mean that as in -10 is smaller than a bunch of numbers in the -5 to 5 range, or do you mean that in absolute value? The latter would imply that half are below and half are above, because the "test differences" should be symmetrically distributed around 0 (without an absolute value). Apr 29 '11 at 13:30