Suppose you have the following model, where all variables are continuous:

$$y = \alpha + \beta_0x_0 + \beta_1x_1 + \beta_2 x_0 x_1 + \epsilon$$

The standard error for the effect of $x_1$ is

$$\text{se}(\hat\beta_1 + \hat\beta_2x_0)= \sqrt{\text{var}(\beta_1) + 2\text{cov}(\beta_1, \beta_2) + \text{var}(\beta_2)x_0^2} $$

The last term in the sum says that as the values of the modulator $x_0$ get large in magnitude the precision of the estimates decreases. This is especially odd if your data is mostly at large magnitudes of $x_0$.

Why is this the case? Is it related to some other properties of linear models or is it a one-off oddity?

  • $\begingroup$ You formula has several typos in it. The correct expression should be the root of the variance of $\hat\beta_1 + x_0\hat\beta_2,$ which immediately shows why the SE varies with $x_0.$ It's unclear why this implies interactions are harder to estimate at high values of $x_0$ -- the SE just as well could decrease as $x_0$ increases through its natural range. $\endgroup$ – whuber Sep 15 '20 at 16:37
  • $\begingroup$ How can the SE decrease? Higher values of $x_0$ increase the last term monotonically as they get larger in magnitude, and that's their only impact on SE. $\endgroup$ – badmax Sep 15 '20 at 16:55
  • 1
    $\begingroup$ It's often the case that the covariance is negative. It can even be far greater than the variance of $\beta_2$ in magnitude. It's also unclear what you mean by "high" values: for instance, is $-1$ higher or lower than $-2$? This issue will be more apparent to you once you fix up the typos in the right hand side of your equation: the covariance needs to be multiplied by $x_0.$ $\endgroup$ – whuber Sep 15 '20 at 18:39
  • $\begingroup$ Thanks, I will add this as an answer. $\endgroup$ – badmax Sep 16 '20 at 0:06

Thanks to whuber who provided the answer in the comments.

The problem was the expression for the standard error was not correct. It should be

$$\text{se}(\hat\beta_1 + \hat\beta_2x_0)= \sqrt{\text{var}(\beta_1) + 2x_0\text{cov}(\beta_1, \beta_2) + \text{var}(\beta_2)x_0^2} $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.