6
$\begingroup$

I have analysed a dataset with a linear regression model, including an interaction term between a binary variable and a continuous variable. The interaction was significant. Afterwards, I have fitted 2 separate models of the continuous variable for each of the 2 groups of the binary variable. The 2 slopes have different signs and one of the two slopes is significant.

I need to calculate the power of the significance of this interaction. I prefer to do this with an R function.

$\endgroup$
2
  • $\begingroup$ Are the power.t.test or power.anova.test what you are looking for? $\endgroup$ – phiver Jul 30 '15 at 7:17
  • $\begingroup$ I think the power.anova.test could be applied if the 2 variables of the interaction are categorical, but in my case, a variable is continuous. $\endgroup$ – Jesus Herranz Valera Jul 30 '15 at 8:30
3
$\begingroup$

I do not think there is an R function to calculate this power. My usual suggestion in such cases is to simulate (e.g., here or here or here). Specifically, pick a sample size $n$, simulate your covariate, your binary variable, the dependence of your outcome on both and the noise. Fit your model and assess whether $p<\alpha$. Do this many times and see how often you detect a significant effect. If you want to determine the necessary sample size to reach a target power, change the $n$ until you reach the power you want.

Yes, this means you have to think about many assumptions (e.g., the distributions of your IVs, whether they are independent, whether linearity makes sense, whether your errors will really be homoskedastic, ...). I would argue that this is a feature, not a bug. You will definitely gain more understanding of your problem than if you found a ready-made power calculator, and it's better to gain this understanding before you run your study than afterwards.

(You are asking before you run your study, for sample size determination, right? So-called "post-hoc power" is meaningless.)

One additional advantage is that the flexibility of this approach allows you to vary your assumptions and analyze whether the power degrades. Or you could simulate other data problems, like missing data etc.

Here is some very simple R code, where I am assuming $n=50$, group membership that is random 50-50, a uniformly distributed covariate that is independent of group membership, and an outcome

$$y=\text{covariate}+\text{groups}*\text{covariate}+\epsilon$$

with $\epsilon\sim N(0,1)$. In this case, power comes out to be 0.166. If you want the standard $\beta=0.80$, you can increase $n$ or change your model assumptions - and think about whether the changed assumptions make sense.

n_sims <- 1000

nn <- 50
alpha <- 0.05
result <- rep(FALSE,n_sims)
pb <- winProgressBar(max=n_sims)
for ( ii in 1:n_sims ) {
    setWinProgressBar(pb,ii,paste(ii,"of",n_sims))
    set.seed(ii)    # for reproducibility

    # this is where the assumptions enter
    covariate <- runif(nn)
    groups <- runif(nn)<0.5
    outcome <- covariate+groups*covariate+rnorm(nn,1)

    # fit the model, determine whether an effect was found
    model <- lm(outcome~groups*covariate)
    p_value <- anova(model,update(model,.~.-groups:covariate))[2,6]
    result[ii] <- p_value<alpha
}
close(pb)

sum(result)/n_sims
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.