1
$\begingroup$

Consider I have a linear model fitted in R as:

model<-lm(y~x*fac)

where x is a continuous variable and fac a sum contrast coded two level factor (-1,1).

now I can check the coefficients of my model and do t-tests to check if:

  1. the main effect /slope of x (averaged over the two factor levels) is significantly different from zero
  2. if the slope for x is different between my two factor levels
  3. what is my intercept averaged over my two factor levels with equal weighting
  4. if the intercepts between factor levels differ from point 3.

What I see is also done often is a type III ANOVA. The car package in R for example has the function car::Anova(model,type="III"). The model output is also providing significance values for x,fac and x:fac. However how can I interpret them? Can I actually interpret them? as I saw that there is a controversy about the "Principle of marginality" for type III ANOVA.

Does this violation of "Principle of marginality" also hold for the interpretation of my coefficient for x (see 1) in the regression itself? Or can I still interpret my slope as the main effect or average slope expected when averaging over my factor levels with equal weights?

Do sum contrast coding and type III ANOVA yield the same results/interpretations (ignoring for a second that one uses F-tests the other t-tests) if I have two levels in my factor, but do those differ in interpretation if my factor has more levels? Or do they overall tell completely different stories? and is one better than the other? I tend to say contrast coding and interpreting coefficients is superior as it also at least gives the magnitude of the effect? -either directly by looking at coefficients or calculating this via the R package emmeans. Does this make sense?

$\endgroup$

1 Answer 1

2
$\begingroup$

In their origin (in SAS), Type III tests are defined via estimable contrasts, and those may or may not correspond to regression coefficients. It seems to me that these tests don't make sense unless you fully understand what those estimable contrasts are. IMHO, it is pretty clear what contrasts are being tested when you are using emmeans::joint_tests(), but not so much with car::Anova(), even though the latter often produces the same results. And joint_tests() can be applied to any set of estimated marginal means, thus allowing you to test different estimable contrasts than those that are mysteriously constructed based on model comparisons.

If you are interested in deciding which models are approriate, Type II tests are a lot more useful as they put a hierarchy on reasonable models.

Addendum

Here's an example for a similar model that makes this more concrete. First, a model with a covariate and a factor interacting and its Type III ANOVA:

> fiber.lm <- lm(strength ~ diameter * machine, data = fiber)
> (jt <- joint_tests(fiber.lm))
 model term       df1 df2 F.ratio p.value
 diameter           1   9  60.996  <.0001
 machine            2   9   2.814  0.1124
 diameter:machine   2   9   0.488  0.6293

We can see the associated estimable functions via an attribute:

> attr(jt, "est.fcns")
$diameter
     (Intercept)  diameter machineB machineC diameter:machineB diameter:machineC
[1,]           0 -0.904534        0        0        -0.3015113        -0.3015113

$machine
     (Intercept) diameter   machineB   machineC diameter:machineB diameter:machineC
[1,]           0        0 -0.0414009  0.0000000        -0.9991426         0.0000000
[2,]           0        0  0.0000000 -0.0414009         0.0000000        -0.9991426

$`diameter:machine`
     (Intercept) diameter machineB machineC diameter:machineB diameter:machineC
[1,]           0        0        0        0                -1                 0
[2,]           0        0        0        0                 0                -1

These are estimable functions of the regression coefficients, and that can be a little confusing to look at. One thing that might make it clearer is to re-parameterize it in terms of the cell means for combinations of machines and a symmetric range of diameters.

> rg <- regrid(ref_grid(fiber.lm, cov.reduce = meanint))
> (jt <- joint_tests(rg))
 model term       df1 df2 F.ratio p.value
 diameter           1   9  60.996  <.0001
 machine            2   9   2.814  0.1124
 diameter:machine   2   9   0.488  0.6293

> # same ANOVA result!

> attr(jt, "est.fcns")
$diameter
     23.1333333333333.A 25.1333333333333.A 23.1333333333333.B 25.1333333333333.B
[1,]         -0.4082483          0.4082483         -0.4082483          0.4082483
     23.1333333333333.C 25.1333333333333.C
[1,]         -0.4082483          0.4082483

$machine
     23.1333333333333.A 25.1333333333333.A 23.1333333333333.B 25.1333333333333.B
[1,]         -0.5000000         -0.5000000          0.5000000          0.5000000
[2,]         -0.2886751         -0.2886751         -0.2886751         -0.2886751
     23.1333333333333.C 25.1333333333333.C
[1,]          0.0000000          0.0000000
[2,]          0.5773503          0.5773503

$`diameter:machine`
     23.1333333333333.A 25.1333333333333.A 23.1333333333333.B 25.1333333333333.B
[1,]         -0.5000000          0.5000000          0.5000000         -0.5000000
[2,]          0.2886751         -0.2886751          0.2886751         -0.2886751
     23.1333333333333.C 25.1333333333333.C
[1,]          0.0000000          0.0000000
[2,]         -0.5773503          0.5773503

Now it is easier to see that weights are balanced across machines and/or diameters, albeit some of the weights are scaled funny.

$\endgroup$
3
  • $\begingroup$ Thanks a lot for your answer and help. I will go with joint_tests then. However can you confirm that interpreting the slope coefficient of the main effect of x (see point 1) is not violating the principal of marginality and can just be interpreted as the slope averaged across the two factor levels? $\endgroup$
    – Jmmer
    Dec 5, 2022 at 22:10
  • $\begingroup$ The type III tests of the main effects of x and of fac definitely do violate the principle of marginality! I also confirm that the associated estimable functions do average with equal weights over the levels of the other factor (the levels of x comprise the endpoints of an interval centered at its mean), so I can't answer either "yes" or "no" to your question. $\endgroup$
    – Russ Lenth
    Dec 6, 2022 at 18:20
  • $\begingroup$ Thanks a lot for this clarification. I will definitely recommend emmeans for that :). So in case x is continuous and fac is a two level factor like in my example and I end up with coefficients say from summary model then I get x, fac1 and x:fac1 in sum contrast coding. Then I will not write x is a "main" effect but rather x is the slope when averaging over fac with equal weights. So more like a general/grant slope across factor levels. with more factor levels this becomes more complicated for the interpretation of fac but the coefficient for x will still be say the grant average across fac. $\endgroup$
    – Jmmer
    Dec 6, 2022 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.