How to uniformly project a hash to a fixed number of buckets Hi Fellow Statisticians,
I have a source generating hashes (e.g. computing a string with a timestamp and other information and hashing with md5) and I want to project it into a fixed number of buckets (say 100).
sample hash:
    0fb916f0b174c66fd35ef078d861a367
What I thought at first was to use only the first character of the hash to choose a bucket, but this leads to a wildly non-uniform projection (i.e. some letters apppear very rarely and other very frequently)
Then, I tried to convert this hexa string into an integer using the sum of the char values, then take the modulo to choose a bucket:
import sys

for line in sys.stdin:
    i = 0
    for c in line:
        i += ord(c)
    print i%100

It seems to work in practice, but I don't know if there are any common sense or theoretical results that could explain why and to which extent this is true ?
[Edit]
After some thought I came to the following conclusion:
In theory you can convert the hash into a (very big) integer by interpreting it as a number : i = h[0] + 16*h[1]+16*16*h[2] ... + 16^31*h[31] (each letter represents an hexadecimal number). Then you could modulo this big number to project it to the bucket space.
[/Edit]
Thanks !
 A: NB: putting in form the answer that emerged from discussion in comments so that it's easier to read for interested people
(updated version)
Suppose we have a source generating independent events that we want to distribute uniformly into $B$ buckets.
The key steps are:


*

*hash each event $e$ to an integer $i$ of size $2^N$

*project onto $\mathcal{R} \times [0, 1[$ as $p = \frac{i}{2^N}$  

*find matching bucket $b_i$ so that $\frac{b_i}{B} \le p \lt \frac{b_{i+1}}{B}$


For 1. a popular solution is to use MurmurHash to generate a 64 or 128 bits integer. 
For 3. a simple solution is to iterate on $j = 1..B$ and check that $p$ is in $[\frac{b_j}{B}, \frac{b_{j+1}}{B}[$
In (python) pseudo-code the overall procedure could be:
def hash_to_bucket(e, B):
    i = murmurhash3.to_long128(str(e))
    p = i / float(2**128)
    for j in range(0, B):
        if j/float(B) <= p and (j+1)/float(B) > p:
            return j+1
    return B


(previous version, really not optimal)
The first observation is that the n-th letter of the hash should be uniformly distributed with respect to the alphabet (which is here 16 letters long - thanks to @leonbloy for pointing that out).

Then, to project it to a [0,100[ range, the trick is to take 2 letters from the hash (e.g. 1st and 2nd positions) and generate an integer with that:
int_value = int(hash[0])+16*int(hash[1])

This value lives in the range [0,16+(16-1)*16[, hence we just have to modulo it to 100 to generate a bucket in the [0, 100[ range:

As pointed out in the comments, doing so impact the uniformity of the distribution since the first letter is more influential than the second.
bucket = int_value % 100

In theory you can convert the whole hash into a (very big) integer by interpreting it as a number: i = h[0] + 16*h[1]+16*16*h[2] ... + 16^31*h[31] (each letter represents an hexadecimal number). Then you could modulo this big number to project it to the bucket space. One can then note that taking the modulo of i can be decomposed into a distributive and additive operation: 
\begin{align}
i \mod N = (&\\
&(h_0 \mod N) \\
&+ (16 \mod N \times h_1 \mod N) \\
&+ ... \\
&+ (16^{31} \mod N \times h_{31} \mod N)\\
&) \mod N 
\end{align}
A: I had a similar problem and came up with a different solution which may be faster and more easily implemented in any language.
My first thought was to dispatch items quickly and uniformly in a fixed number of buckets, and also to be scalable, I should mimic randomness.
So I coded this little function returning a float number in [0, 1[ given a string (or any kind of data in fact).
Here in Python:
import math
def pseudo_random_checksum(s, precision=10000):
    x = sum([ord(c) * math.sin(i + 1) for i,c in enumerate(s)]) * precision
    return x - math.floor(x)

Off course it's not random, in fact it's not even pseudo random, the same data will always return the same checksum. But it acts like random and it's pretty fast.
You can easily dispatch and later retrieve items in N buckets by simply assigning each item to bucket number math.floor(N * pseudo_random_checksum(item)).
A: Here you can find a a branchless uniform bucket distribution that works with with bitwise operations.
