# Would using the md5 for treatment assignment introduce any biases?

Disclaimer: I'm not sure if there's a better stackexchange for this, but since I consider experimental design an integral part of causal/statistical inference, I'm hoping this is an appropriate audience.

So, I'm considering setting up a controlled experiment on a service that has users (each have unique user_ids). To determine which users get put into which group ("control" or "treatment"), an engineer has proposed the following assignment algorithm:

if (md5(user_id) mod 100 < control_fraction)
assign user_id to control
else
assign user_id to treatment


Now, I've empirically determined that for $X \in \mathbb{N}$, $$md5(X) \mod 100 \sim \mathcal{U}nif[0,100)$$ Here's the distribution of values when $X \in \{1,2,\ldots,100k\}$

But I'm still left with lingering doubts as to whether or not this assignment mechanism may introduce some sort of subtle selection/enrollment biases...

Does this scheme allow me to proceed with my experimental analysis, as if we had assigned users randomly, or do I need to take anything into account?

• Any random number generated by a computer is only a deterministic algorithm that _behaves quite like_a random variable. This one looks good enough to me (but I am too new to statistics to formulate it as an answer). Aug 4, 2015 at 4:33

## 1 Answer

This question has been sitting here a while, but better late than never.

There is a very slight bias with the given approach. Applying the md5 algorithm will result in a uniform distribution in the 128 bit output space. However, that means there are $2^{128}$ possible outputs. $2^{128} \equiv 56\space (\text{mod } 100)$, so you're slightly more likely to get users in the 0-56 range and less likely to get users in the 57-99 range than you should be.

At the end of the day, this only means that more users could end up in the control group than expected. (Although assuming you have at most 7.3 billion users, this issue is very unlikely to manifest.)

Is there a reason you can't simply use a random number instead of modding an md5?

• Welcome to our site - note that you can use Latex formatting here by putting text between dollar signs, e.g. $2^{128}$ produces $2^{128}$ Sep 13, 2016 at 21:38