24
$\begingroup$

We have a dataset with three variables (dV: self-reported measure on scale 1-5, assumed to be metric; iV1: factor with 4 levels; iV2: factor with 8 levels). We are interested whether the dV differs in regard to both iVs and whether there is an interaction between the iVs.

Idea: Calculating an ANOVA with both main effects and the interaction between both iVs using R.

The question: What Type of Sum of Squares should be used for this research question?

Using aov() in R calculates Type-I Sum of Squares as standard. SPSS and SAS, on the other hand, calculate Type-III Sum of Squares by default. However, using Anova() {car} in combination with options(contrasts=c("contr.sum", "contr.poly")) in R gives the same Type-III ANOVA tables as calculated in SPSS.

I have already read the following discussions:

However, I am still confused which Type of Sums of Squares is the most adequate for our question. The results (F and p values) differ considerably.

$\endgroup$
  • $\begingroup$ If you read the discussions you'd find that both correlations among the predictors and balance in the designs impact which one of the types of SS to use. Please flesh out those details in your question. $\endgroup$ – John May 29 '13 at 21:19
  • $\begingroup$ The design is unbalanced and one missing cell occurs. Concerning 'correlation' among the factors: chisq.test(iV1, iV2) gives a p < .001. Do you need more information? $\endgroup$ – phx May 30 '13 at 8:11
  • $\begingroup$ p isn't the concern...r is...maybe kramer's phi, or even the chi. $\endgroup$ – John May 30 '13 at 17:31
  • 1
    $\begingroup$ Main effects are not universally uninterpretable if there are significant interactions. For one of many possible examples, if A in Y ~ A * B is relatively large compared to all other effects, and especially if B is known to be of limited range (e.g. sex) then interpeting A is OK, maybe even important, even if A:B is significant. (And conversely, the mere presence of an interaction, whether significant or not, makes regression coefficients for main effects take on limited meaning.) $\endgroup$ – John Apr 8 '14 at 22:15
  • 2
    $\begingroup$ Extremely related: stats.stackexchange.com/questions/20452 with a great answer by @gung. $\endgroup$ – amoeba Apr 29 '16 at 22:06
17
$\begingroup$

I realize this is a year old post but its likely to come up again.

There are many factors that play into this, I'd argue the most important is your hypothesis. So there is no clear answer but I generally follow these rules-of-thumb:

1) Type II is only when you don't have an interaction term.

2) Type I vs Type III to test the interaction term...I go Type I all the time reason is this

Type I SS for dv ~ A + B + A*B depends on order so...its sequential SS(A) SS(B|A) SS(A*B|A B)

This is great to test your interaction...but not great to test the main effects since the effect of B is dependent on A.

Type II gets around this SS(A|B) SS(B|A)

Which looks great to test your main effects IF there is no interaction term.

Type III: SS(A| A*B B) SS(B| A*B A)

Which given the most common hypotheses...doesn't seem very useful since most people are interested in the interaction term, not the main effects when an interaction term is present.

So in this case I'd use Type-I to test the interaction term. If not significant I'd refit without the interaction term and use Type-II to test the main effects.

warning: anova() in R is Type-I, to get Type-II (or III) use the Anova() in the car package.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ +1 for pointing out the hypothesis definitely matters. I was probably too focused on the specific problem in my answer. For the simple cases you're describing Type I and Type II are the same with respect to an interaction. The only difference would be with multiple interactions the Type I is still sequential but the Type II takes into account the other interactions, like it does with main effects. $\endgroup$ – John Dec 6 '16 at 16:53
10
$\begingroup$

I've never been a fan of Type III SS for ANOVA's so this is a biased recommendation.

I believe you should select Type II in this case. In the Type I ANOVA order matters. So, whether you include iV1, or iV2 first makes a difference because the first (e.g. iV1) is compared to a model with just an intercept while the second is compared to a model with an intercept and the first. Try out switching the order they're in and you'll see the difference in main effect outcomes because your predictors are correlated.

The Type III gets around this by assessing each predictor, including the interaction against a model including everything but that predictor. That sounds like a good idea until you try to consider what an interaction is without one of the main effects included. You're comparing the predictor to what is essentially a nonsensical model (I think one of your references sort of goes into this). (Furthermore, my recollection is that Type III is especially sensitive to missing cells (thus Type IV I believe).)

Type II gets around the order issue in Type I and compares sensible models (unlike Type III). Main effects are tested with all other main effects in the model but not the interaction. Thus each main effect is easily interpreted as the unique contribution of that predictor.

Note, all of the SS types discussed would get the same interaction effect in your case because it's assessed with all of the main effects included in each case.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks. What about the problem that Type II assumes that the interaction term is non-existing? Is it possible to calculate a Type IV ANOVA in R? I couldn't find any information on that. $\endgroup$ – phx May 31 '13 at 15:06
  • $\begingroup$ What about the argumentation of this paper: yourpsyche.org/sites/default/files/InteractionsAndTypesOfSS.pdf? The author argues that Type I ANOVAS only test hypothesis concerning the sample and not the population. $\endgroup$ – phx May 31 '13 at 15:47
  • 1
    $\begingroup$ You said in your question comments that you wanted the Type to use if no interaction is assumed. This is the Type. It's the best one for main effects if there is no interaction. If there is an interaction it may have biased main effects but that won't matter much because the interaction will be the most important thing and you can pretty much ignore main effects (unless they are very large, in which case it doesn't matter much if they're a little biased). $\endgroup$ – John May 31 '13 at 17:08
  • $\begingroup$ The Type II "assumption" of no interaction is perhaps an exaggeration of the term. It's mainly descriptive in distinction from Type III. When you test main effects in Type III you're including (nonsensical) interactions while in Type II you leave them out, thus no interaction. You can test for an interaction and, whether it's there or not, feel free to go on and test for main effects. (drop1 does Type II in R). No crime will have been committed. Similarly Type I tests main effects with no interaction as well. $\endgroup$ – John Apr 8 '14 at 21:59
3
$\begingroup$

I have recently made a decision : I will never use again an ANOVA sum-of-squares testing, except for the interaction. Why ?

  • Because, in general, the hypothesis $H_0$ of the tests of the main effects are diffcult to interpet.

  • Because we can do something really more instructive and interpretable: multiple comparisons with confidence intervals.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ @phx Are you able to write down the $H_0$ hypothesis for the main effects tests ? (in the non-balanced case). Yes I mean simultaneous confidence intervals. With a balanced design, detecting a significant difference for at least one comparison should be equivalent to get a significant result for the corresponding ANOVA test. In the non-balanced case, multiple comparisons are always easy to interpret, whereas ANOVA is diffuclt, and the confidence intervals provide much more information than an ANOVA table. Moreover sometimes you are not interested in all pairwise comparisons. $\endgroup$ – Stéphane Laurent May 31 '13 at 16:37
  • $\begingroup$ Thanks. You mean to calculate multiple comparisons using alpha corrections? Problems might occur if you have factors with large (k>3) number of levels. Why do you think the H0 is hard to interpret? $\endgroup$ – phx May 31 '13 at 22:06
  • $\begingroup$ I have to think about the concrete $H_0$. Multiple comparisons might be an approach to follow. However, at the moment we are asking the question whether iV1 (4 levels) has influence on y controlling for iV2 (8 levels). I'm not sure whether multiple comparisons are appropriate in this context. $\endgroup$ – phx May 31 '13 at 22:11

Not the answer you're looking for? Browse other questions tagged or ask your own question.