1
$\begingroup$

I am trying to learn about statistical hypothesis testing. As an example problem I want to study the effect of rep range on muscle growth. N subjects will be randomly assigned into one of two groups:

  • High (H) load: each workout consists of 3 sets x 5 repetitions of leg press.
  • Low (L) load: each workout consists of 3 sets x 10 repetitions of leg press.

The percentwise increase in cross sectional area of the leg muscles over the training period will be recorded.

My hypotheses are:

  • H_0: there will be no difference in percentwise increase across group.

  • H_A: the Low group will have larger percentwise increase: $\mu_L >= 1.1 \mu_H$

I choose a 5% significance level.

What is the minimum value of N?

How do I perform the the test?

I just need some pointers.

$\endgroup$

1 Answer 1

1
$\begingroup$

First, I suppose your data will be normal. You have said you want to do a one-sided test at the $\alpha = 5\%$ level, hoping to detect a difference in group means of size $\Delta = 0.1.$ That is, you will test $H_0: \mu_H - \mu_L = 0$ against $H_a: \mu_H - \mu_L < 0.1.$ [I think I understand your intentions, but your statement of null and alternative hypotheses is not quite right.]

In order to find the required sample size $n$ for each group, you would need to state the desired power of the test and to estimate the variance of the observations.

  • Because you are willing to make Type I error (falsely rejecting when $H_0$ is true) with probability $\alpha = 0.05 = 5\%,$ perhaps you want Type II error (failing to reject when $\mu_a = 1.1)$ with probability $\beta = 0.05$ also, which means you want power $1 = \beta = 0.95 = 95\%.$ (That is, you want to be $95\%$ sure to detect a difference as large as $\Delta.$

  • Unless you have previous experience with such experiments it may be difficult to estimate the variance of the observations. The variability may be smaller if the subjects are of roughly equal fitness going into the experiment. For purposes of illustration, I will take $\sigma^2 = (0.2)^2 = 0.04.$

A common practice is to provide (at least tentatively) values of $\alpha, \Delta,$ and $\sigma^2$ and then to use a 'power and sample size' procedure to balance affordable sample size $n$ (for each group) with achievable power. For pooled 2-sample t tests, an exact computation requires use of a _noncentral Student's t distribution. [There are also 'power and sample size' procedures (of varying degrees of reliability) available online.]

_ However, required sample sizes are often above $n = 30$ and then the following approximate formula gives a useful approximate result. For my speculative values, $n \approx 87.$

$$n \approx \frac{2\sigma^2(z_\alpha + z_\beta)^2}{\Delta^2} =\frac{2(0.2^2)(1.645+1.645)^2}{0.1^2} \approx 87.$$

The discussion above is for pooled 2-sample t tests, which assume that the two groups have equal variances. Unless you have prior knowledge or experience with such measurements, it will be best to use a Welch 2-sample t test. If the two groups do have equal variances, then power will be about the same as for the pooled two-sample t test. Otherwise, power may be a little less and you will be protected against systematically incorrect results.

Using two normal samples simulated to the specifications discussed above, here is how a Welch two-sample, one-sided t test would look in R. [Because I'm using a Welch t test I increased the sample sizes to 100 in each group---as a precaution.]

set.seed(829)  # for reproducibility
x1 = rnorm(100, 1, .2)
x2 = rnorm(100, 1.1, .2)
summary(x1); length(x1); sd(x1)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.5226  0.8466  0.9983  1.0005  1.1275  1.5506 
[1] 100
[1] 0.2068055
summary(x2); length(x2); sd(x2)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.4899  0.9561  1.0734  1.0572  1.1858  1.5258 
[1] 100
[1] 0.185094
boxplot(list(x1,x2), col="skyblue2", label=T)

enter image description here

t.test(x1, x2, alt="less")

        Welch Two Sample t-test

data:  x1 and x2
t = -2.0403, df = 195.61, p-value = 0.02133
alternative hypothesis: 
  true difference in means is less than 0
95 percent confidence interval:
       -Inf -0.0107568
sample estimates:
mean of x mean of y 
 1.000535  1.057161 

The following simulation in R shows that with means $\mu_1 = 1, \mu_2 = 1.1$ and other parameters specified above, the Welch two-sample, two-sided t test rejects with power about 97%, so perhaps I was a little too cautious using sample sizes $n = 100$ instead of $n = 87$ as estimated by my approximate sample-size equation.

set.seed(2020)
pv = replicate(10^5, t.test(rnorm(100,1,.2), rnorm(100,1.1,.2),alt="less")$p.val)
mean(pv <= 0.05)
[1] 0.97027

A similar simulation with sample sizes $n = 87$ does give about 95% power---with no adjustment needed.

Addendum per Comments: If you have reason to believe your data might not be normal, you should consider the use of nonparametric tests. However, you asked about comparing population 'means', and nonparametric tests often do not compare means. To assume routinely from the start that data are not normal may sometimes be prudent, but may sometimes be foolish and counterproductive.

$\endgroup$
4
  • $\begingroup$ Not clear why you started out with the normality assumption. $\endgroup$ Aug 29, 2020 at 21:12
  • $\begingroup$ Chapter 7 of Biostatistics for Biomedical Research hbiostat.org/bbr $\endgroup$ Aug 30, 2020 at 13:35
  • $\begingroup$ Full document is at hbiostat.org/doc/bbr.pdf $\endgroup$ Aug 30, 2020 at 17:20
  • $\begingroup$ "In case data are not normal" assumes that the sample size is large enough that non-normality would be detected. Best to just allow for non-normality. $\endgroup$ Sep 1, 2020 at 2:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.