Why include insignificant main term when interaction term is included? [duplicate]

Question

Most of the statistics material suggest that when the interaction term is significant, even though the main effect term is not significant, we still keep the main effect term.

For example: $$z = a + b\cdot x + c\cdot y + d\cdot xy$$

$x$ is continuous while $y$ is categorical (two levels, $y_1$ and $y_2$). $z$ is continuous. The linear model fitted has a significant $xy$ term, intercept term and $x$ term but non-significant $y$ term.

Can I reduce the model to $z = a' + b'\cdot x + d'\cdot xy$ ? Interpretation is that for level $y_1$ and level $y_2$, they have the same intercept but different slope.

Thank you!

Answer 1

That violates the hierarchy principle and is highly dependent on the choice of origins for the measurements (e.g., change temperature from F to C and you'll get a much different model). It is not appropriate to look at statistical tests of "main effects" when interactions are present.

Answer 2

If you think of a regression as Taylor expansion, then dropping the main effect is equivalent to imposing a very strong constraint on the model. Here's the intuition.

Say, the model is $$y(x_1,x_2)=\beta_0+\beta_1x_1+\beta_2x_2+\beta_{11}x_1^2+\beta_{22}x_2^2+\beta_{12}x_1x_2$$

You can see that the betas correspond to Taylor expansion coefficients (Maclaurin, actually), e.g.: $$\frac{\partial^2 y}{\partial x_1\partial x_2}=\beta_{12}$$

So, when you fit this model to the data, you're pretty much approximating the unknown function by its Taylor expansion.

Now, when you drop the main effect, you're imposing the following constraint: $\beta_1=0$. In terms of Taylor expansion you're saying that $\partial y/\partial x_1=0$. This is a very strong constraint on the shape of your model. For instance, if you're using only the second order expansion, then you're saying that the function is symmetrical around 0.

Is it right to impose such a constraint? It depends on what you know about the underlying phenomenon. If you don't know anything about its functional form, then it's better not to drop the main effect.

Answer 3

Permit me to simplify your question.

In a simple ANOVA model such as:

y(i,k) = grand.mean + treatment(i) + error(i,k)

would you remove the grand.mean if it does not test significantly different from zero and fit just:

y(i,k) = treatment(i) + error(i,k)?  

If not, why not? One good reason is that the second model CONDITIONS your test for treatment(i)==0 on the PRESUMPTION that grand.mean==0 while the first model does not. A pretty good reason not to condition in this way is that failure to reject grand.mean<>0 does not imply grand.mean==0!