0
$\begingroup$

I have a question regarding the output from a discrete-time survival model that I have previously asked questions about [here][1].

The code looks the following (see link further up for more info):

mod<-glm(Mating_time ~ Round + A*B,data=dat, family=binomial(link="cloglog"))

Anova(mod)

> Anova(mod)
Analysis of Deviance Table (Type II tests)

Response: as.factor(Mating_time)
                        LR Chisq Df Pr(>Chisq)    
Round                     35.094  8  2.571e-05 ***
A                          3.749  2    0.15343    
B                          9.024  2    0.01097 *  
A : B                     10.424  4    0.03385 *  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> 

multcomp::cld(emmeans(Mating_time, ~A|B), Letters = letters, reversed = T) # Post-Hoc test

> multcomp::cld(emmeans(Mating_time, ~A|B), Letters = letters, reversed = T) # Post-Hoc test
B = Metabolite:
 A           emmean    SE  df asymp.LCL asymp.UCL .group
 Normal       -1.87 0.218 Inf     -2.29     -1.44  a    
 Metabolite   -1.98 0.192 Inf     -2.35     -1.60  a    
 Low          -1.99 0.217 Inf     -2.42     -1.57  a    

B = Normal:
 A           emmean    SE  df asymp.LCL asymp.UCL .group
 Low          -1.49 0.213 Inf     -1.91     -1.08  a    
 Normal       -2.35 0.249 Inf     -2.84     -1.86   b   
 Metabolite   -2.59 0.264 Inf     -3.10     -2.07   b   

B = Low:
 A           emmean    SE  df asymp.LCL asymp.UCL .group
 Normal       -2.26 0.230 Inf     -2.71     -1.81  a    
 Low          -2.55 0.265 Inf     -3.07     -2.03  a    
 Metabolite   -2.62 0.236 Inf     -3.09     -2.16  a    

Results are averaged over the levels of: Round 
Results are given on the cloglog (not the response) scale. 
Confidence level used: 0.95 
Note: contrasts are still on the cloglog scale 
Results are given on the as.factor (not the response) scale. 
P value adjustment: tukey method for comparing a family of 3 estimates 
significance level used: alpha = 0.05 
NOTE: If two or more means share the same grouping symbol,
      then we cannot show them to be different.
      But we also did not show them to be the same. 

I know that ANOVA and post-hoc tests such as Tukey's test can give different output since they are different tests that answers different questions. However, the output I get is somewhat confusing to me. I do see significance of B and A : B interaction (also round but that's less important to what I'm actually interested in, I'm interested in how A and B and their interaction influences mating_time). Due to the way we structured our research question, we do pairwise comparisons by B (three comparisons for the three groups formed). However, the output I get from the ANOVA says that there's significance for B and the interaction A : B. However, when I do the post-hoc test I would suspect all comparisons being the same because of this. Although I see differences in the "B normal" group, this I would expect to be the case if A on ANOVA showed significance. I can't wrap my head around how this could be the case if just B and A : B interaction showed significance.

EDIT:

When I try to run type 1 SS anova (either with anova() command or aov()) on the model I run into the following error that I can't find no correction for online.

> test<-aov(mod)
Warning messages:
1: In model.response(mf, "numeric") :
  using type = "numeric" with a factor response will be ignored
2: In Ops.factor(y, z$residuals) : ‘-’ not meaningful for factors
> summary(test)
Error in levels(x)[x] : only 0's may be mixed with negative subscripts

Works with Anova type II and III though. Any thoughts?

$\endgroup$
2
  • $\begingroup$ aov() is a function for fitting a model. I think you meant anova() or car::Anova() $\endgroup$
    – Russ Lenth
    Commented Jul 27, 2023 at 17:40
  • $\begingroup$ I tried with both aov() then summary on the aov model that I created as seen above. Should do the trick if I'm not mistaken but I get the error code. Also for the anova() I get the same error as for aov(). With Car::Anova() I can only do type II or III anova and not type I as with anova(), hence the reason I try to run it. You mentioned in an earlier comment that type II anova ignore the interaction $\endgroup$
    – Blanca
    Commented Jul 28, 2023 at 8:01

2 Answers 2

0
$\begingroup$

A significant interaction means that the combination of two treatments has a different effect than you would predict based on the effects of the treatments individually. In the calculations, the interaction term is represented by the product of the predictor values, as coded.

Interactions are thus inherently symmetric. You can express the interaction equivalently as either "the effect of B depends on the level of A" or "the effect of A depends on the level of B." In your type of display, you might expect to find something like what you show regardless of whether you evaluated levels of A within levels of B (as you show) or the other way around.

In your data, A has no effect when B = Metabolite or B = Low. You could predict those values just on the basis of the value of B. For those treatment combinations, the value of A doesn't matter.

When B = Normal, however, you can't predict the combinations with either an effect of B (ignoring A) alone, or an effect of A (ignoring B) alone. That's why you had a significant interaction term: there were situations in which you couldn't adequately predict the outcomes based on separate effects of A and B.

$\endgroup$
0
$\begingroup$

Your Anova results express type 2 tests. That means that the test of A is based on the model with just A and B, and not A:B. That is, when you completely ignore the interaction of A and B, the marginal effect of A is not all that big.

I also suggest that you spend more effort looking at what you have and less effort on shopping for P values. A graph like emmip(Mating_time, A ~ B) should be helpful for understanding what is going on.

$\endgroup$
5
  • $\begingroup$ Thanks for the input on the ANOVA, so I would rather go for either type I or III. I've plotted the data according to the emmip(Mating_time, A ~ B). Would you suggest to look at contrasts to see where the effects are located in a following manner: mod.int <- emmeans(mod, ~A*B) # look at contrasts mod.int I'm asking since if I'm interested in the interaction effect, I should have some statistics to report the least. In my field, people seems to include interaction effects in the model (and run type 2 anovas as I have seen) but don't report statistics further than ANOVA output. $\endgroup$
    – Blanca
    Commented Jul 25, 2023 at 17:22
  • $\begingroup$ I would do emm.int <- emmeans(mod, ~A|B) then pairs(emm.int) and pairs(emm.int, by = "A"). In other words, simple comparisons of A for each B and of B for each A. If people in your field don't report things past anova output, then shame on them because they are star-gazing for *'s but not bothering to try to understand what they have. $\endgroup$
    – Russ Lenth
    Commented Jul 25, 2023 at 19:47
  • $\begingroup$ Thanks, so a possible way to report the results from the comparisons of A for each B and B for each A in a figure could be to use capital letters for lets say A for each B and small letters for B in each A? I'm creating figures based on the estimated marginal means and the SE given by the model. $\endgroup$
    – Blanca
    Commented Jul 26, 2023 at 9:41
  • $\begingroup$ Another comment on the subject, i looked into this forum post link where you suggested looking at contrasts of contrasts, in the case I have an interaction effect, this would also be suggested right? From my knowledge, code you wrote above looks at A for each B and vice versa. But since I also have the interaction effect, after running ANOVA (type III) I'll go for comparisons suggested above and the contrasts of contrasts to see where the interaction effects are present? $\endgroup$
    – Blanca
    Commented Jul 26, 2023 at 12:29
  • $\begingroup$ Made an edit in the post regarding the type I ANOVA. I receive an error while running it. This doesn't happen with type 2 or 3 from Anova() command though. $\endgroup$
    – Blanca
    Commented Jul 27, 2023 at 11:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.