How to do you find a power of a test of two means?

Question

I am struggling with the following question

Suppose we have randomly assigned 50 patients to the new treatment group and the other 50 patients to the conventional treatment group. Assuming both distributions follow normal distribution, with means µ1 and µ2, respectively, with standard deviations of the two groups are assumed to be 1.

The researcher expects to see a difference in the means ∆ = µ1 − µ2 = 0.6. She plans to employ a simple one-sided test for the two groups comparison at the significance level α = 0.05. What is the power of detecting such a difference based on the sample she has?

I know the power is probability of rejecting the null hypothesis if the null hypothesis is false But I am not sure how to proceed at all from here? How do we know any probability here. Any help to get me started is much appreciated

Answer 1

This can actually be framed as a linear regression, so let's use a power formula found in this book.

The power to detect an effect of $\beta$ is

$$ \gamma = 1 - \Phi(z_{1-\alpha/2} - \vert \beta \vert\sigma_x\sqrt{n}\sigma_{y \vert x})$$

Here

• $\Phi$ is the CDF of the standard normal.

• $z_{1-\alpha/2} \approx 1.96$ if we are using the standard false positive rate

• $\beta=0.6$ as per your question

• $\sigma_x=\sqrt{0.5^2} = 0.5$. This is because the groups are evenly split.

• $\sigma_{y\vert x}=1$ is standard deviation of the outcome.

• $n=100$ as per your question

Plugging this all in R, it looks like there is about 85% percent power to detect an effect as small as 0.6.

You can also use simulation to verify the results

library(purrr)
library(tidyverse)

set.seed(0)
sim<-function(){
  x = rnorm(50,0.6,1)
  y = rnorm(50,0,1)

  return(t.test(x,y, equal.var = T)$p.value<0.05)
}

rerun(10000,sim()) %>% unlist %>% mean

Looks like we correctly reject the null about 84.9% of the time according to this simulation, which is exactly what we would expect (up to random error).

Answer 2

Power is computed as follows:

Step 1: Figure out your rejection rule under the null.

Step 2: Identify the distribution of the test statistic under the specific alternative.

Step 3. Find the probability of being in the rejection region defined in (1), given the distribution in (2).

- I assume you don't need any help with step 1.

- Since the population standard deviations are assumed known (and equal to 1) you should be able to write the distribution of the test statistic (for a z-test) when the means differ by 0.6. That is, step 2 doesn't require much effort.

- Once step 2 has been accomplished, I assume you don't need any help with step 3.