0
$\begingroup$

I would like to calculate an a-priori power analysis for a within-groups study with the following design:

  • participant_id
  • condition (within groups - A, B)
  • covariate (scale 1-7)
  • questionnaire items (scale 1-7)
  • binary_outcome

I generated a following fake dataset like so

# Set the number of participants
num_participants <- 50

# Create a data frame for participants
participants <- data.frame(participant_id = 1:num_participants)

# Generate a random order for A and B for each participant
participant_order <- sample(c("A_first", "B_first"), num_participants, 
      replace = TRUE)

# Create a data frame with two rows per participant
dummy_data <- data.frame(participant_id = 
    rep(participants$participant_id, each = 2),
    condition = rep(c("A", "B"), times = num_participants),
    order = rep(participant_order, each = 2))

# Add a covariate on a 7-point scale
dummy_data$covariate <- sample(1:7, num_participants * 2, 
    replace = TRUE)

# Simulate differences between conditions
set.seed(123)  # For reproducibility

for (item in 1:3) {
  dummy_data[paste("item", item, sep = "_")] <- 
    rnorm(num_participants * 2, 
    mean = ifelse(dummy_data$condition == "A", 4, 6), sd = 1)
}

# Simulate differences for the binary outcome "bin_1"
dummy_data$bin_1 <- rbinom(num_participants * 2, size = 1, 
prob = ifelse(dummy_data$condition == "A", 0.3, 0.7))

dummy_data <- dummy_data %>%
  dplyr::mutate(
    item_TOTAL = rowMeans(dplyr::select(., item_1, item_2, item_3)))

This specifies a difference between condition A and B on item_TOTAL. For the power analysis I am using the simr function, so I created a mixed model using lme4.

model1 <- lmer(item_TOTAL ~ condition + covariate + 
               (1|participant_id), data = dummy_data)
summary(model1)

This gives me a fixed effect of 1.87 for conditionB. Following the tutorial here I can change the fixed effect size like so

model2 <- model1 
fixef(model2)["conditionB"] <-- 0.05

I can then perform power analyses using the powerSim function

powerSim(model1)
powerSim(model2)

The code so far all seems to be working, however I am confused about editing the parameters for the actual power analysis. The tutorial linked mentions "specifying an effect size", and that `the estimated effect size for x is -0.11, which is significant at the .01 level using the default z-test.'

But as I understand it, the fixed effect coefficients are not the same as an effect size, so I am not sure how I should be changing the fixef coeffecient to correspond to different expected effect sizes (since I imagine this is also design dependent). As the data is randomly generated to begin with I am also not sure what the fixef(model2)["condition"] <-- 0.05 command is actually doing - should first I generate random data with no effect and then only edit fixef? Or should I generate 3 fake data sets, each with a different effect size (which I am not even quite sure how to do this) and then compare the 3? I would also in real life expect a positive correlation between my covariate and outcome variable, should this also be accounted for somehow?

I am also not sure what the benefits of doing a simulated power analysis are over just treating the design as a repeated measures t-test and calculating the power that way. Any advice is appreciated!

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.