Power of the t-test under unequal sample sizes

Question

I heard that the power of a t-test used with unequal samples is limited by the size of the smaller sample. Can I take this to mean that the power of a t-test with unequal sample sizes is equal to the power of a t-test used on equal sample sizes where $n$ equals the size of the smaller sample?

Answer 1

(Note, by $n$, I usually mean the total sample size, so I interpret your last sentence to be 'where $\bf{.5}$$n$ equals the size of the smaller sample'.)

No, not quite. Consider this simulation (conducted with R):

set.seed(9)

power1010 = vector(length=10000)
power9010 = vector(length=10000)

for(i in 1:10000){

  n1a = rnorm(10, mean=0,  sd=1)
  n2a = rnorm(10, mean=.5, sd=1)

  n1c = rnorm(90, mean=0,  sd=1)
  n2c = rnorm(10, mean=.5, sd=1)

  power1010[i] = t.test(n1a, n2a, var.equal=T)$p.value
  power9010[i] = t.test(n1c, n2c, var.equal=T)$p.value
}

mean(power1010<.05)
[1] 0.184

mean(power9010<.05)
[1] 0.323

What we see here is that when the total sample size is $20$, with equal group sizes, $n_1=n_2=10$, power is $18\%$; but when the total sample size is $100$, but the smaller group has $n_2=10$, power is $32\%$. Thus power can increase when the size of the larger group goes up even though the smaller sample size stays the same.

This answer is adapted from my answer here: How should one interpret the comparison of means from different sample sizes?, which you will probably want to read for more on this topic.

Answer 2

To understand the comment, consider the effect of letting the second sample get larger and larger while the first stays constant in size. Eventually, the sample mean for the second sample converges to the population mean it was drawn from, and the standard error of the mean becomes zero. If you examine the ordinary two-sample test statistic

$$t = \frac{\bar {X}_1 - \bar{X}_2}{S_p \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$$

then as $n_2$ gets large, $\bar{X}_2$ goes to $\mu_2$, and the term in the square root goes to $1/n_1$.

Now look at $S_p$:

$$S_p = \sqrt{\frac{(n_1-1)S_{X_1}^2+(n_2-1)S_{X_2}^2}{n_1+n_2-2}}$$

Let $w_1 = \frac{n_1-1}{(n_1-1)+(n_2-1)}$ and similarly for $w_2$

$$S_p = \sqrt{w_1 S_{X_1}^2+w_2 S_{X_2}^2}$$

As $n_2$ gets large, $w_2$ goes to 1, while $S_{X_2}$ goes to $\sigma_2$; $S_p$ becomes $\sigma_2$.

So what are we left with? The statistic now looks like this:

$$ \frac{\bar {X}_1 - \mu_2}{\sigma_2 / \sqrt{n_1}}$$

An ordinary one-sample z-test ... whose power is a function of $n_1$.

You might find it instructive to consider the same calculation in the case of the Welch t-test (I believe you get a one-sample t statistic).