2
$\begingroup$

I need to conduct a power analysis, but I'm not sure if I'm doing it correctly. Initially, I have four units, and each unit will be divided into two halves. In Units 1 and 2, stimulation (S) will be applied, while in Units 3 and 4, there will be no stimulation (NS). Each half of these units will receive one of two treatments: Treatment 1 (T1), which is the control, and Treatment 2 (T2). The specific application of these treatments is shown in the figure.

diagram

In Unit 1, Treatment 1 will be applied to Side 1 and Treatment 2 will be applied to Side 2. For Unit 2, the treatment arrangement is reversed. Units 3 and 4 will follow the same pattern of alternating treatments on their respective sides.

In my experiment, a force is applied (via a machine) and measured to each side of all the units. The idea being that I need to measure the force required to cut into the units. The objective is to test the following hypotheses:

  • $H_0:$ There is no difference between the force applied to treatments 1 and 2 ($T_1 = T_2$).

  • $H_1:$ The force applied with treatment 1 is greater than that applied with treatment 2 ($T_1 > T_2$).

Then, I need to test whether there is a difference in Treatment 2 with and without stimulation.

  • $H_0:$ There is no difference in the applied force between treatment 2 with or without stimulation ($T_2 \times S = T_2 \times S$).

  • $H_1:$ The force applied with treatment 2 and stimulation is less than that applied with treatment 2 without stimulation ($T_2 \times S < T_2 \times S $).

I have the means, standard deviations, and sample sizes from articles that I found in the literature, but I'm not sure how to calculate the effect size and how to determine the correct sample size (considering a power of $0.80$ and a significance level of $a = 0.05$). In my example, I have four units, but I need to work out how many times I need to repeat the experiment (i.e, how many times I need to repeat these groups of four units).

I initially calculated this using the pwr.anova.test function from the pwr package in R, but I'm not confident if my results are correct. Can anyone suggest any methods that I could use to work out the sample size? Or do the methods I'm using sound correct?

$\endgroup$

1 Answer 1

2
$\begingroup$

This looks like a two-way ANOVA disguised as a three-way ANOVA. The critical things I see that you are testing are these two factors:

  • The effect of stimulation.
  • The effect of treatment.
  • (Optional) The interaction between the two.

I originally believed that unit here was another factor, but that is irrelevant because you will always have units split up in this way if you have a two-way ANOVA. Because you are already conditioning for order effects in both stimulation settings (by using the first treatment first, then in reverse), there is no reason to account for this (though if you didn't have a more randomized order here then it would potentially be a problem).

Since you are using R, there is another package that is built for two-way ANOVA power simulations that is fairly painless. The key ingredient is knowing what $F$ or $\delta$ values you are anticipating or expecting for these factors. Below I write an example with a case where you know what you want/expect. Note that if you are looking for sample size and not power, you can use the ss.2way function, which has similar arguments:

#### Load Library ####
install.packages("pwr2") # if not already installed
library(pwr2)

#### Function Call ####
pwr.2way(
  a = 2, # number of groups in Factor A
  b = 2, # number of groups in Factor B
  alpha = .05, # alpha cutoff criterion
  size.A = 50, # number of people in Factor A
  size.B = 20, # number of people in Factor B
  f.A = .3, # anticipated/desired effect size Factor A
  f.B = .4, # anticipated/desired effect size Factor B
  sigma.A = .1, # standard deviation of Factor A
  sigma.B = .4 # standard deviation of Factor B
)

The output is shown below (note that the power is derived from the minimum power obtained from each group):

     Balanced two-way analysis of variance power calculation 

              a = 2
              b = 2
            n.A = 50
            n.B = 20
      sig.level = 0.05
        power.A = 0.9881484
        power.B = 0.9421146
          power = 0.9421146

NOTE: power is the minimum power among two factors

If you don't know what sample sizes or effect sizes you expect, you can use the power plot functions in this package to derive a range of sample sizes and effect sizes to guide that decision. As an example:

#### Setup Parameters ####
n <- seq(2, 30, by=4) # seq of sample sizes (from 2 to 30)
f <- seq(0.1, 1.0, length.out=5) # sequences of effect sizes (.1 to 1)

#### Check Power Plot ####
pwr.plot(
  n=n, # total number of people
  k=2, # number of groups
  f=f, # anticipated/desired effect size 
  alpha=0.05 # alpha cutoff
  )

You can see how the effect size and sample size shape the power curves here:

enter image description here

The legend is a bit annoying to work around, but if you run a sequence and are interested in that particular curve, you can just run that by itself:

#### Setup Parameters ####
n <- seq(2, 30, by=4) # seq of sample sizes (from 2 to 30)
f <- .8  # just one effect size

#### Check Power Plot ####
pwr.plot(
  n=n, # total number of people
  k=2, # number of groups
  f=f, # anticipated/desired effect size 
  alpha=0.05 # alpha cutoff
  )

enter image description here

You can see from this individual curve that with a very strong effect size, around $10$ people is sufficient for your power cutoff of $.80$ (I doubt you would ever get this in the social sciences but its an extreme example for demonstration purposes).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.