2
$\begingroup$

Say we have a well setup A\B test that puts half of users in one arm, half in another.

Do we describe the sample space as $\Omega_1$ for arm1 and $\Omega_2$ for arm2? Or do we instead just say the outcomes of the experiment are over a single sample space, $\Omega$?

I'm trying to formalize my test as a series of elements that map to Bernoulli r.v.'s from two distributions with unknown parameters $p_1$, $p_2$.

However, it's not clear to me how the concepts of a sample space are reconciled with experiment design. Are the outcomes simply encoded such that we map random variables for each arm to the same outcome space? Or do we instead say that we have two sample spaces that are characterized by different distributions? I'm struggling a little with the language here.

$\endgroup$

1 Answer 1

3
$\begingroup$

It is the same sample space (like $\Omega=\{0,1\}$ or $\{H,T\}$), but we calculate the probability of seeing the observed effect (or something more extreme) under the null that the two groups are drawn from the same distribution (i.e., $B(p_1)=B(p_2)=B(p)$) or at least that the two distributions have the same mean or sometimes also variance.

$\endgroup$
2
  • $\begingroup$ What does it mean to be "drawn" from a distribution? $\endgroup$ Commented Sep 30, 2023 at 5:36
  • 1
    $\begingroup$ Drawn means sampled. $\endgroup$
    – dimitriy
    Commented Sep 30, 2023 at 5:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.