linear regression with interaction term

Question

I have a Pearson correlation of r=.30 between two variables - x and y. Now, I want to see whether this relationship changes as a function of variable z. I then calculated the following regression y~x+x:z (using lm in R). The standardized beta coefficient for the effect of x jumps to 1.07, and the x:z interaction coefficient is -.70. I am a bit worried that these numbers are unrealistic. That is, given that the relationship between x and y is low, it sounds too good to have such a high standardized regression coefficient. Any reason I should be worried about this analysis? The correlation between x and z is r-.06. I don't know if that means anything.

Thank you, Nitzan.

Answer 1

Due to the principle of marginality, the unconditional correlation between $x$ and $y$ is not very meaningful in the presence of an interaction. The value does not tell you anything about the plausibility of the coefficients obtained in the linear model with interaction.

Quoting Wikipedia:

The principle of marginality implies that, in general, it is wrong to test, estimate, or interpret main effects of explanatory variables where the variables interact [...].

A simple example in R shows that the numbers you obtained are not very surprising at all:

set.seed(1234)
n <- 1000
x <- rnorm(n)
z <- rnorm(n, 1)
y <- x -0.7 * x * z
cor(x, y)
> 0.3219056
cor(x, z)
> 0.05663004

In this example, there is a near zero correlation between $x$ and $z$, a correlation of about $0.3$ between $x$ and $y$ and a true effect of:

$$y = 1 \cdot x + 0 \cdot z - 0.7 \cdot xz$$

when considering both $x$ and $z$ simultaneously.

Be careful though, you shouldn't try to model y ~ x + x:z, since this also violates the principle of marginality:

[...] or, similarly, to model interaction effects but delete main effects that are marginal to them.

The correct way to test an interaction is to include both $x$ and $z$ as follows:

y ~ x * z
y ~ x + z + x:z # same as above