5
$\begingroup$

I've been taught this general rule: power to detect significant interaction effects in a model is smaller than the power to detect significant main effects in a model. I've never really understood why this is. Why is the power to reject a null hypothesis for an interaction smaller than the power to reject a null hypothesis for a main effect? My only thought is that it is due to a loss of degrees of freedom that come with interaction models, but I'm wondering if there's more to it than that.

$\endgroup$
5
$\begingroup$

It's signal and noise. When looking for an interaction, you're looking for it against more noise.

Consider looking for a mean in a single group. Only the noise of that group affects your estimate of the mean.

Now consider looking for a difference of means between two groups. The estimated difference is affected by the independent noise of both groups. The difference (the signal) is subject to more noise, so we are less likely to reliably detect it.

Now consider looking for a difference of differences of means, that is, a two-way interaction. That involves the noise from four independent groups. The difference of differences is subject to yet more noise, so we are even less likely to reliably detect it.

That's a quick conceptual answer. Others may want to provide more technical details...

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I like the intuition of this answer. I'm curious if there's a way to formalize it more technically, perhaps by mixing standard errors or some such approach. $\endgroup$ – Ashe Apr 5 '17 at 19:35
1
$\begingroup$

Here's the pure intuition.

Consider these two functions: $$f(x_1,x_2)=\beta_0+\beta_1x_1+\beta_2x_2$$ vs. $$g(x_1,x_2)=\beta_0+\beta_1x_1+\beta_2x_2+\beta_{12}x_1x_2$$

You need just three observations to solve for the parameters of $f(x_1,x_2|\beta_0,\beta_1,\beta_2)$, but you need four observations of function $g(x_1,x_2|\beta_0,\beta_1,\beta_2,\beta_{12})$ because it has one more parameter.

The function $g(.)$ represents a model with an interaction term, and function $f(.)$ a model with just the main effects. Even if you forget about the random noise, you see that $g(.)$ needs more data than $f(.)$ to establish its parameters.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This appears to be formalizing the loss of degrees of freedom that accompanies a typical interaction model? If that's true, then does it logically follow that the difference between power of the interaction and power of the main effect gets smaller as N gets larger (ie the difference in degrees of freedom between models gets proportionally smaller)? $\endgroup$ – Ashe Apr 5 '17 at 19:36
  • $\begingroup$ I wasn't going after degrees of freedom argument. It's more of an intuition around saying that "it's easier to see the slope than the bend". $\endgroup$ – Aksakal Apr 5 '17 at 20:12
0
$\begingroup$

I don't think this is a statistical question as much as a question of the state of the world. In most realistic situations, differences between effects are smaller than the effects themselves. However, this is not statistically or logically necessary.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.