ANOVA with and without interactions giving different values for main effects

Question

I have measured Copper content in fishing nets. I have 2 independent variables - treatment of the net with 4 levels and type of net with 2 levels. I'm using R to do the ANOVA.

I care about the interaction, so I should do:

mCu = aov(Cu ~ Type * Treatment, data = Ultrasonic)

But why do I get different values for the difference in type if I do:

mCu = aov(Cu ~ Type + Treatment, data = Ultrasonic)

I thought using the asterisk instead of the plus sign would just add the interactions as well, not change the analysis of the variables by themselves. (Although, the values are not very far off.)

Answer 1

The issue is that when you include an interaction term, the meaning of the main effects changes. Specifically, what would be called the "main effects" in an additive model are really just the effects when the other interacting variable is at the reference level in an interaction model.

To help you understand these topics better, here are some of my other answers that discuss relevant issues:

• What does “all else equal” mean in multiple regression?
• How can logistic regression have a factorial predictor and no intercept?
• Interpretation of betas when there are multiple categorical variables
• Interpretation of interaction term

Answer 2

I had this very same doubt myself. You didn't say if you are using repeated measures or single measures. Lets assume you are using repeated measures. If you use single measures and you take the interaction term out of the error there would be no other source of variation so there would be no error term (the error term in the single-measure multi-factor anova is exactly the interaction term).

So, the total variation, measured with the sum of squares, is

$SS_{Total} = SS_{Factor1} + SS_{Factor2} + SS_{Interaction} + SS_{Error}$

If you assume that the interaction term is significant you are assuming that the $SS_{Interaction}$ term is a systematic variation and that the casual error, or residual or variation due to casuality, is just $SS_{Error}$.
So when you calculate the F-values for factors you use the statistics $\frac{MS_{Factor1}}{MS_{Error}}$ and $\frac{MS_{Factor2}}{MS_{Error}}$.

In the other case you are assuming that the interaction is not significant and that the variations measured with the $SS_{Interaction}$ term are due just to the casuality of sampling. In this case the masure of the error is given by $SS_{Interaction}$+$SS_{Error}$.
So the statistics you use in this case are $\frac{MS_{Factor1}}{MS_{Error+Interaction}}$ and $\frac{MS_{Factor2}}{MS_{Error+Interaction}}$. Where $MS_{Error+Interaction}=\frac{SS_{Error}+SS_{Interaction}}{d.f._{Error}+d.f._{Interaction}}$.

You can do this because

$E(MS_{E+I}) = E(\frac{SS_{E}+SS_{I}}{d.f._{E}+d.f._{I}}) = E(\frac{SS_{E}}{d.f._{E}+d.f._{I}})+E(\frac{SS_{I}}{d.f._{E}+d.f._{I}}) = E(\frac{SS_{E}}{d.f._{E}}\frac{d.f._{E}}{d.f._{E}+d.f._{I}})+E(\frac{SS_{I}}{d.f._{I}}\frac{d.f._{I}}{d.f._{E}+d.f._{I}}) = E(\frac{SS_{E}}{d.f._{E}})\frac{d.f._{E}}{d.f._{E}+d.f._{I}}+E(\frac{SS_{I}}{d.f._{I}})\frac{d.f._{I}}{d.f._{E}+d.f._{I}}$.
Under the hypothesis of non significant interactions $E(\frac{SS_{I}}{d.f._{I}})=\sigma^2$ so $E(MS_{E+I}) = \sigma^2 \cdot (\frac{d.f._{E}}{d.f._{E}+d.f._{I}}+\frac{d.f._{I}}{d.f._{E}+d.f._{I}}) = \sigma^2$.

So at the denominator of $\frac{MS_{Factor}}{MS_{Error+Interaction}}$ you have a variable, as you had in the first case, whose expected value does not depend on the differences between means and the ratio only accounts for the numerator part that grows larger as the differences between means grow larger.

So the procedure is as follows:

1. run the anova including interactions; in you example mCu = aov(Cu ~ Type * Treatment, data = Ultrasonic)
2. check if the interactions are significant at a given significance level according to their p-value
3. if they are significant you are done with the anova table you have (some sources recommend in this case to recalculate the F-value for factors using $\frac{MS_{Factor}}{MS_{Interaction}}$ instead of $\frac{MS_{Factor}}{MS_{Error}}$ because factor differences are meaningful only if the are large compared to factor interactions)
4. if they are not significant rerun the anova considering the interaction variation as part of the error term, that means mCu = aov(Cu ~ Type + Treatment, data = Ultrasonic)

You can verify when you run these that the d.f. of the residuals of the + experiment are equal to the sum of the d.f. of the residuals and the d.f. of the interaction of the * experiment and that the sum of squares of the residuals of the + experiment is equal to the sum of the sum of squares of the residuals and the sum of squares of the interaction of the * experiment.