Optimal unbalanced design for A/B test I want to find the right way to distribute variants (control/test) in A/B-test with the constraint of no more 10 percent of population in test group. The test is t-test for continuous metric.
I can design in 2 different ways:


*

*Split control/test in 90/10 percent proportion, and calculate the evaluation metric on test (10% of population) and full control (90%).

*Expose experiment on 20% of population, 10% on control and 10% on test.  Other 80% will not take part in metric calculation.


Which is the best from the view of statistical power?
 A: @DemetriPananos correctly states that, with a 2-sample t test, power for a given
total sample size $(n_1+n_2)$ is maximized when subjects are split with equal numbers in the two groups $(n_1=n_2).$ Consequently, the 'power and sample size' procedures in many statistical software programs give results
only for balanced designs (equal sample sizes).
However, if you have a constraint on the number in the treatment
group and it is not vastly expensive to use a larger control group, then you might consider using a larger control group, which would give you some
increase in power.
I illustrate for the specific case of distinguishing
between a treatment group sampled from $\mathsf{Norm}(\mu = 105,
\sigma = 5)$ and a control group from 
$\mathsf{Norm}(\mu = 100, \sigma = 5).$ So we are trying to
design an experiment that can often detect a difference in
population means that is equal to one SD.
The following simulations in R show that the power is reasonably good (about 87%) with
$n = 20$ in each group, but noticeably better (about 98%) with $n_1 = 20$
in the treatment group and $n_2=200$ in the control group.
set.seed(2020)
pv.bal = replicate(10^5, t.test(rnorm(20,105,5), rnorm(20,100,5))$p.val)
mean(pv.bal <= .05)
[1] 0.86832   # aprx power for n1 = n2 = 20

pv.unb = replicate(10^5, t.test(rnorm(20,105,5), rnorm(200,100,5))$p.val)
mean(pv.unb <= .05)
[1] 0.98289   # aprx power for n1 = 20, n2 = 200

Notes: (1) Many 'power and sample size' procedures give results
for pooled 2-sample t tests, rather than the generally preferable Welch 2-sample t tests, which do not assume equal variances in treatment
and control groups. 
(2) The simulations shown above are for Welch 2-sample t tests. The power advantage of a larger
control group might be usefully magnified if the control group had a larger variance than the treatment group.
set.seed(609)
pv.bal = replicate(10^5, t.test(rnorm(20,105,4), rnorm(20,100,6))$p.val)  # Trt var 16, Ctrl var 36
mean(pv.bal <= .05)
[1] 0.85287
pv.unb = replicate(10^5, t.test(rnorm(20,105,4), rnorm(200,100,6))$p.val)
mean(pv.unb <= .05)
[1] 0.99812

(3) In the R code vectors such as pv <= .05 are logical 
vectors consisting of TRUEs and FALSEs. The mean of
such a vector is its proportion of TRUEs. With 100,000
iterations, power values should be accurate to two or more
decimal places.
A: The power for such a test is roughly (according to the end of chapter 4 of this book) roughly...
$$ \beta = 1-\mathbf{\Phi}^{-1}(z_{1-\alpha/2}-\Delta\sqrt{f(1-f)}\sqrt{n}/\sigma_y) $$
Here, $\Delta$ is the smallest meaningful effect, $f$ is the fraction of people in the test group, $n$ is the sample size, and $\sigma_y$ is the standard deviation of the outcome.  The only thing that differs between your two designs is $f$ and $n$.  We want $\Delta\sqrt{f(1-f)}\sqrt{n}/\sigma_y$ to be as big as possible because that results in the most power.  All else considered equal, we only have to worry about $f$ and $n$.
In the first case $f=0.1$.  In the second, $f=0.5$ but we have a 20% of our sample.  We need to determine, which is larger: $.5  \times \sqrt{0.2 \times n}$, or $0.3 \times \sqrt{ n}$?  
Turns out...
$$0.5  \times \sqrt{0.2 \times n} < 0.3 \times \sqrt{ n}$$ 
So the first design (more controls, smaller exposure) should yield more power. How much more power will depend on other parameters.  The difference could be very big or very small.
Here is some code to double check my arguments
library(tidyverse)


case_a = mean(replicate(1000,{

  Ny = 10
  Nx = 90
  x = rnorm(Nx)
  y = rnorm(Ny, 1)
  t.test(x,y)$p.value<0.05

}))
>>>0.765

case_b = mean(replicate(1000,{

  Ny = 10
  Nx = 10
  x = rnorm(Nx)
  y = rnorm(Ny, 1)
  t.test(x,y)$p.value<0.05

}))
>>>0.58

You could very easily draw power curves by either computing my first equation, or varying the size of the difference between the two samples in these simulations.
