9
$\begingroup$

I heard that the power of a t-test used with unequal samples is limited by the size of the smaller sample. Can I take this to mean that the power of a t-test with unequal sample sizes is equal to the power of a t-test used on equal sample sizes where $n$ equals the size of the smaller sample?

$\endgroup$
1
  • $\begingroup$ If you perform the power of the t-test in R, for example using power_t_test from MESS library, it is true that the smallest sample size is required, but a sample sizes ratio is required too to deal with unequal sample sizes t-test. Sample sizes ratio conditionates power result. $\endgroup$ Mar 7, 2023 at 12:59

2 Answers 2

16
$\begingroup$

(Note, by $n$, I usually mean the total sample size, so I interpret your last sentence to be 'where $\bf{.5}$$n$ equals the size of the smaller sample'.)

No, not quite. Consider this simulation (conducted with R):

set.seed(9)

power1010 = vector(length=10000)
power9010 = vector(length=10000)

for(i in 1:10000){

  n1a = rnorm(10, mean=0,  sd=1)
  n2a = rnorm(10, mean=.5, sd=1)

  n1c = rnorm(90, mean=0,  sd=1)
  n2c = rnorm(10, mean=.5, sd=1)

  power1010[i] = t.test(n1a, n2a, var.equal=T)$p.value
  power9010[i] = t.test(n1c, n2c, var.equal=T)$p.value
}

mean(power1010<.05)
[1] 0.184

mean(power9010<.05)
[1] 0.323

What we see here is that when the total sample size is $20$, with equal group sizes, $n_1=n_2=10$, power is $18\%$; but when the total sample size is $100$, but the smaller group has $n_2=10$, power is $32\%$. Thus power can increase when the size of the larger group goes up even though the smaller sample size stays the same.

This answer is adapted from my answer here: How should one interpret the comparison of means from different sample sizes?, which you will probably want to read for more on this topic.

$\endgroup$
2
  • $\begingroup$ You're welcome, @Jimj. You might want to check out the other answer as well. $\endgroup$ Jul 10, 2013 at 2:02
  • $\begingroup$ That's right, @John, I discuss that in the other answer that I link to. My only point here is that: the power of a t-test w/ unequal sample sizes is not equal to the power of a t-test w/ equal sample sizes where .5n equals the size of the smaller sample. $\endgroup$ Jul 10, 2013 at 2:59
6
$\begingroup$

To understand the comment, consider the effect of letting the second sample get larger and larger while the first stays constant in size. Eventually, the sample mean for the second sample converges to the population mean it was drawn from, and the standard error of the mean becomes zero. If you examine the ordinary two-sample test statistic

$$t = \frac{\bar {X}_1 - \bar{X}_2}{S_p \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$$

then as $n_2$ gets large, $\bar{X}_2$ goes to $\mu_2$, and the term in the square root goes to $1/n_1$.

Now look at $S_p$:

$$S_p = \sqrt{\frac{(n_1-1)S_{X_1}^2+(n_2-1)S_{X_2}^2}{n_1+n_2-2}}$$

Let $w_1 = \frac{n_1-1}{(n_1-1)+(n_2-1)}$ and similarly for $w_2$

$$S_p = \sqrt{w_1 S_{X_1}^2+w_2 S_{X_2}^2}$$

As $n_2$ gets large, $w_2$ goes to 1, while $S_{X_2}$ goes to $\sigma_2$; $S_p$ becomes $\sigma_2$.

So what are we left with? The statistic now looks like this:

$$ \frac{\bar {X}_1 - \mu_2}{\sigma_2 / \sqrt{n_1}}$$

An ordinary one-sample z-test ... whose power is a function of $n_1$.

You might find it instructive to consider the same calculation in the case of the Welch t-test (I believe you get a one-sample t statistic).

$\endgroup$
2
  • $\begingroup$ In the equation for the ordinary two-sample test statistic, is it a typo to put the pooled standard deviation Sp as part of the denominator? $\endgroup$
    – vtshen
    Mar 25, 2020 at 5:14
  • $\begingroup$ No. $\!$$\!$$\!$$\!$ $\endgroup$
    – Glen_b
    Mar 25, 2020 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.