7
$\begingroup$

I heard that the power of a t-test used with unequal samples is limited by the size of the smaller sample. Can I take this to mean that the power of a t-test with unequal sample sizes is equal to the power of a t-test used on equal sample sizes where $n$ equals the size of the smaller sample?

$\endgroup$
13
$\begingroup$

(Note, by $n$, I usually mean the total sample size, so I interpret your last sentence to be 'where $\bf{.5}$$n$ equals the size of the smaller sample'.)

No, not quite. Consider this simulation (conducted with R):

set.seed(9)

power1010 = vector(length=10000)
power9010 = vector(length=10000)

for(i in 1:10000){

  n1a = rnorm(10, mean=0,  sd=1)
  n2a = rnorm(10, mean=.5, sd=1)

  n1c = rnorm(90, mean=0,  sd=1)
  n2c = rnorm(10, mean=.5, sd=1)

  power1010[i] = t.test(n1a, n2a, var.equal=T)$p.value
  power9010[i] = t.test(n1c, n2c, var.equal=T)$p.value
}

mean(power1010<.05)
[1] 0.184

mean(power9010<.05)
[1] 0.323

What we see here is that when the total sample size is $20$, with equal group sizes, $n_1=n_2=10$, power is $18\%$; but when the total sample size is $100$, but the smaller group has $n_2=10$, power is $32\%$. Thus power can increase when the size of the larger group goes up even though the smaller sample size stays the same.

This answer is adapted from my answer here: How should one interpret the comparison of means from different sample sizes?, which you will probably want to read for more on this topic.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ You're welcome, @Jimj. You might want to check out the other answer as well. $\endgroup$ – gung - Reinstate Monica Jul 10 '13 at 2:02
  • $\begingroup$ That's right, @John, I discuss that in the other answer that I link to. My only point here is that: the power of a t-test w/ unequal sample sizes is not equal to the power of a t-test w/ equal sample sizes where .5n equals the size of the smaller sample. $\endgroup$ – gung - Reinstate Monica Jul 10 '13 at 2:59
6
$\begingroup$

To understand the comment, consider the effect of letting the second sample get larger and larger while the first stays constant in size. Eventually, the sample mean for the second sample converges to the population mean it was drawn from, and the standard error of the mean becomes zero. If you examine the ordinary two-sample test statistic

$$t = \frac{\bar {X}_1 - \bar{X}_2}{S_p \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}$$

then as $n_2$ gets large, $\bar{X}_2$ goes to $\mu_2$, and the term in the square root goes to $1/n_1$.

Now look at $S_p$:

$$S_p = \sqrt{\frac{(n_1-1)S_{X_1}^2+(n_2-1)S_{X_2}^2}{n_1+n_2-2}}$$

Let $w_1 = \frac{n_1-1}{(n_1-1)+(n_2-1)}$ and similarly for $w_2$

$$S_p = \sqrt{w_1 S_{X_1}^2+w_2 S_{X_2}^2}$$

As $n_2$ gets large, $w_2$ goes to 1, while $S_{X_2}$ goes to $\sigma_2$; $S_p$ becomes $\sigma_2$.

So what are we left with? The statistic now looks like this:

$$ \frac{\bar {X}_1 - \mu_2}{\sigma_2 / \sqrt{n_1}}$$

An ordinary one-sample z-test ... whose power is a function of $n_1$.

You might find it instructive to consider the same calculation in the case of the Welch t-test (I believe you get a one-sample t statistic).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ In the equation for the ordinary two-sample test statistic, is it a typo to put the pooled standard deviation Sp as part of the denominator? $\endgroup$ – vtshen Mar 25 at 5:14
  • $\begingroup$ No. $\!$$\!$$\!$$\!$ $\endgroup$ – Glen_b Mar 25 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.