# Calculating sample size when you have 2 treatments and uneven distribution

I have 2 new conversion webpages to test. Leadership recommends 80% traffic to existing conversion webpage and 10% each to the 2 new conversion webpages (non-negotiable)

How do I calculate sample size? I understand the fundamentals of sample size calculation - alpha, power, baseline conversion rate and minimum detectable effect needed but have done it for a 50/50 distribution and 1 treatment only..

I am unsure how to calculate for 2 treatment and uneven distribution. Can anyone guide me or refer me to a calculator or formula?

• What null and alternative? What test statistic? May 18, 2023 at 3:46

Main point: use a standard RCT methodology but adjust your $$\alpha$$ level accordingly to account for multiple comparisons.
More details: Irrespective of how many arms we have and their respective sizes, it is still the case of the smaller sample sizes dominating the power of the study - smaller sample sizes inflate the occurrence of Type II errors. That means we need to run our experiment longer (to get more data) and/or make the different arms of the study have "more dramatic" changes (to get larger expected effect sizes). On top of that, we need to do a correction for multiple comparisons in our $$\alpha$$ too because we now have two comparisons, i.e. use a family-wise error rate for an adjusted level of $$\frac{\alpha}{k}$$ is usually fine ($$k$$=2 here). Notice that this latter point is counter to the usual idea of increasing $$\alpha$$ to avoid Type II errors. If one wants to explore this behaviour a bit further using R, the function stats::p.adjust is immediately available in every R installation and offers a good place to start.