0
$\begingroup$

I have the dataset where the dimension = 10 and number of samples = 20. Let's denote the features by $x_1, x_2, ..., x_{10}$. I'd like to analyze the effect of $x_2$ on $x_1$. I applied the following concept which is similar to A/B testing to solve this problem.

First, I clustered the samples into two groups based on their values of $x_2$, with those above the $median(x_2)$ being in group A and others in group B. In order to reduce the bias, I also want both group A and B to be randomized groups. That is, for each group, I want features $x_3, x_4, ..., x_{10}$ to be as diverse (randomized) as possible. To use the analogy in A/B testing, if $x_3$ is age, I want group A can have samples ranging from 10 years old to 70, and likewise for group B. Last, for features other than $x_1$ or $x_2$, there should be minimal differences between group A and group B at the aggregated level.

The above reasoning can be simplified into the following objectives:

Form group A and B from the dataset where

  1. $avg(x_{A2}) - avg(x_{B2})$ is maximized
  2. $avg(x_{Ai}) - avg(x_{Bi})$ is minimized for $i = 3, 4, ..., 10$

After the groups are formed, we can calculated some summary statistics of $x_1$ like $avg(x_1)$ for each group to see whether they are different.

I am wondering if there are better ways to form the above groups. I tried to simply plot boxplots for each group, and swap samples to make both groups similar in terms of $x_3, ..., x_{10}$, but this is not a systematic way. Although the effect of $x_2$ on $x_1$ can be found simply by regressing $x_1$ on all other features, or by some pairwise statistics like correlation coefficients or mutual information, I wonder how it can be solved via the above approach. Any comment will be appreciated. Thanks!

$\endgroup$
0
$\begingroup$

There are two methods for A/B-testing: stratification and randomization. The idea of the first is— you just select the users by parameters you think matters for your test: age, gender, location, etc. And the idea of randomization— you just select the users randomly, at some moment your sample group will be a very similar to your general population. That's it.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.