I need to compare two targeting models in an experiment. We will apply the same treatment to everyone who is targeted.
The experimental set up is as follows:
- The total population is split into three groups: A (10K), B (10K), C (4k)
- We then rank the population of Group A according to algorithm A and apply a treatment to the top 5k people.
- We rank the population of Group B according to algorithm B, and apply the same treatment to the top 5k.
- In Group C we do no ranking and apply no treatment.
My question is:
- How do we compare the success of each group?
- Do we compare the total number 'cured' after receiving the treatment in Groups A vs Group B?
- Do we compare the total 'cured' in A vs B (both compared to the control group C)?