# any difference between randomly selecting before or after an event?

Is there a mathematical difference between randomly selecting a population of users before an event vs after an event, for the purposes of experimentation?

For example, let's say we want to split all users that perform event X into two groups, A and B. A is 90% of the users (randomly selected) and B is 10% of the users.

Is there any difference if we first randomly split all users into two groups, A and B, at 90% and 10% respectively, and then of all those users, those who perform event X, they will then be sorted based on their already-membership in group A or B?

• Your example is unclear. You have event A and group A. So when you say A is 90% of the users, which A is being referred to. Btw, why would you use the same letter to describe two separate entities in an example? Jul 12, 2016 at 20:25
• sorry about that, I didn't proofread. I initially wrote 90%/10% but then thought to simplify it with A and B, but then forgot I used A for the event name.
– Ste
Jul 12, 2016 at 20:31
• The descriptions of "sorting" and "splitting" are obscure in this question. Could you provide a small example to show what it is you are contemplating?
– whuber
Jul 12, 2016 at 20:59
• @whuber, sorry that's my fault. I meant sorting and splitting to mean the same thing but I wasn't sure exactly how it should be worded. In the above example, what I meant to say was: once a user performs event X, then afterwards, all the participants would then be put into the same A and B buckets that they were selected in before the event X.
– Ste
Jul 12, 2016 at 22:02

I will answer with a cautious No.

Imagine that I select 100 volunteers to play a carnival game. They must make a basketball shot. That is our event X. Let's say we know that the average person makes the shot 60% of the time.

Scenario 1 - Divide first then shoot

90 of the volunteers go to group A and 10 are assigned to group B. 60% of the volunteers make the shot as expected. Leaving us with 54 (90*0.6) in A and 6 (10*0.6) in B.

Scenario 2 - Shoot first then divide

All 100 volunteers shoot the basket and 60% make the shot as expected. Group A will have 54 and B will have 6.

Conclusion

No difference. The same probability distribution results. A smarter person will be able to prove this mathematically. But the crucial bit is that membership to a group must, in itself, not alter the probability of event X occurring.

• Thank you. So if there is any bias in terms of how event X were achieved, for example, basketball players would be naturally selected for, then that would alter the results and the grouping would be different?
– Ste
Jul 12, 2016 at 22:12
• Yes. If there is something different about the groups that will affect the outcome then you group prior to the event. Jul 12, 2016 at 22:14
• The two are slightly different in the possibly irrelevant sense that in Scenario 2, exactly 90 percent will be assigned Group A by construction while in Scenario 1, on average 90 percent will be assigned Group A. But the two are basically the same in the sense that if assignment to Group A is random, it will be independent of event $X$ and all other predetermined unobservables in either scenario (unless of course event $X$ is somehow a function of assignment to group A). Jul 13, 2016 at 0:14
• As Pierre mentioned, "the crucial bit is that membership to a group must, in itself, not alter the probability of event X occurring." For example, if Group A eats a big tub of ice cream, they may get a belly ache and miss free throws at a higher rate (i.e. event X is less likely to occur if assignment to Group A occurs first). Jul 13, 2016 at 0:25