# What A/B test calculator can I use to calculate the minimum same size for uneven split between control and treatment groups?

I have only come across A/B test calculators that give the minimum sample size per variation for 50-50 split between control and treatment.

I am running an A/B test for conversion rates with a 80/20 split between control and treatment. I need to calculate the minimum sample size needed for statistical significance. Can anyone point me to the calculator that does so for uneven split between control and treatment?

Example calculator i have found so far :'https://www.evanmiller.org/ab-testing/sample-size.html' . This calculator is only for 50-50 split.

• Please describe what about the problem made a hypothesis test (test for existence of a signal) appropriate, as opposed to estimating the magnitude of an effect (with sample size chosen to achieve a certain precision in the estimate, i.e., margin of error). Jun 30, 2019 at 12:40

From the point of view of the 'power' of the test, the most efficient design is to have equal sample sizes in A and B, so it is customary for software programs and online calculators to assume that $$n_1 = n_2$$ and then give you the number required in each group to achieve a given power for detecting a given difference in means.

In your case, I suggest you take the answer to be the size of the treatment group, because the power will depend on that more crucially than on the size of the larger control group. ("A chain is only as strong as it's weakest link.")

If you could say what kind of test your will do, it might be possible to give a formula based on both $$n_1$$ and $$n_2.$$ If not a formula, then a simulation method.

According to the power you want about your test, here the table that gives you everything: https://en.wikipedia.org/wiki/Sample_size_determination#Estimating_sample_sizes

Good question, answered by Lachin in his book Biostatistical Methods.

Assuming your AB test is in terms of a binary outcome, the sample size formula is

$$N = \Big( \dfrac{Z_{1-\alpha} \phi_0 + z_{1-\beta}\phi_1} {\Delta} \Big)^2$$

Here,

• $$\Delta$$ is the expected difference between groups.
• $$\phi_0 = \pi (1-\pi)\times 6.25$$ is the variance component under the null. $$\pi$$ is the expected success rate under the null.
• $$\phi_1 = 5\times\pi_1(1-\pi_1) + 1.25\times \pi_2(1-\pi_2)$$ is the variance component under the alternative. $$\pi_1$$ is the expected success rate in the 20% exposure, $$\pi_2$$ is the expected success rate in the 80% exposure group.

I'm not aware of any calculators which implement this, but this is easily done in R or excel, or even pen and paper frankly.