1
$\begingroup$

I have a within-subject dataset with 3 two-level factors and 1 numeric predictor. I was using LMER with a random-intercept model (a full random-effect model yield the same results though throws convergence warnings)

Crucially, one of the hypothesized interactions comp*corr is far from being significant with t = 0.6. The contrasts are set to contr.sum and with type 3 Anova test I will get p = 0.53 for that interaction.

summary(lmer_fit2<-lmer(awaren ~ (comp+corr+distance+eccentr)^4+(1|date), data = awaren_bs))
...

Fixed effects:
                              Estimate Std. Error t value
(Intercept)                   0.105062   0.015198    6.91
comp1                         0.004870   0.009879    0.49
corr                          0.898071   0.013970   64.28
distance1                    -0.021856   0.009879   -2.21
eccentr                       0.030428   0.002744   11.09
comp1:corr                   -0.008639   0.013970   -0.62
comp1:distance1               0.008253   0.009879    0.84
comp1:eccentr                 0.005976   0.002744    2.18
corr:distance1                0.037073   0.013970    2.65
corr:eccentr                 -0.045072   0.003878  -11.62
distance1:eccentr             0.022817   0.002744    8.31
comp1:corr:distance1         -0.008255   0.013970   -0.59
comp1:corr:eccentr           -0.006192   0.003878   -1.60
comp1:distance1:eccentr      -0.002895   0.002744   -1.05
corr:distance1:eccentr       -0.034998   0.003878   -9.03
comp1:corr:distance1:eccentr  0.003038   0.003878    0.78

However, when I'm doing the same analysis with bayesFactor, I get a totally different result:

> anova_res_1<-generalTestBF(awaren~(comp+corr+distance+eccentr)^4+date, awaren_bs, whichRandom = 'date', neverExclude=c('date'),whichModels = 'top')
  |======================================================================================================================================| 100%
> anova_res_1
Bayes factor top-down analysis
--------------
When effect is omitted from comp + corr + distance + eccentr + comp:corr + comp:distance + comp:eccentr + corr:distance + corr:eccentr + distance:eccentr +     comp:corr:distance + comp:corr:eccentr + comp:distance:eccentr +     corr:distance:eccentr + comp:corr:distance:eccentr + date , BF is...
[1] Omit comp:corr:distance:eccentr : 9.096641      ±42.71%
[2] Omit corr:distance:eccentr      : 1.895645e-16  ±45.61%
[3] Omit comp:distance:eccentr      : 10.74473      ±42.71%
[4] Omit comp:corr:eccentr          : 6.881117      ±42.79%
[5] Omit comp:corr:distance         : 17.17485      ±43.47%
[6] Omit distance:eccentr           : 0.4269731     ±46.48%
[7] Omit corr:eccentr               : 8.269684e-27  ±42.86%
[8] Omit corr:distance              : 2.288794e-15  ±42.82%
[9] Omit comp:eccentr               : 4.673529      ±42.43%
[10] Omit comp:distance              : 6.021156      ±42.74%
[11] Omit comp:corr                  : 0.02781902    ±42.67%
[12] Omit eccentr                    : 0.005301817   ±45.12%
[13] Omit distance                   : 0.04890581    ±42.77%
[14] Omit corr                       : 6.956763e-431 ±40.56%
[15] Omit comp                       : 0.5002814     ±44.02%

The removal of comp*corr interaction leads to a worse model meaning that it is "significant".

I have several questions:

1) Do I translate LMER model to BayesFactor model correctly?

2) Do I understand correctly that type 3 ANOVA and generalTestBF with whichModels = "top" should give more or less the same results?

3) Why might I get such a discrepancy between the results?

I've uploaded the data, the output, and the minimal working example here: https://drive.google.com/open?id=0ByJtKXU-AjqmVHNQTHN5eEQ2elE

$\endgroup$

1 Answer 1

1
$\begingroup$

Short story: interaction wiped out the main effects and main effects are in presence of numerical predictor the effects when numerical predictor = 0 because sum-to-zero contrasts have no effect on numerical predictors.

Long story: Turns out that if I use LMER with only main effects and two-factor interactions, the effect of comp*corr is also significant there. It disappears when a third-order interaction of comp1:corr:eccentr is added. So apparently this is a case when interaction "wipes out" the main effect (this question helps to understand how it can happen) despite the sum-to-zero contrasts. An sum-to-zero contrasts obviously have no effect on numerical predictors so that the effects not including numerical predictor (eccentr) are the effects at eccentr=0. So they are not "main effects" in a sense of the "an effect of the variable controlling for everything else" but simply contrasts at the baseline level.

Mystery solved.

So the discrepancy in the results arises because of the different meaning of "main effects" in LMER (main effect is the contrast when numerical predictor = 0) an BayesFactor (comparing explanatory power of the models with and without the effect). Note that neither of the two interpretations is "correct", but both of them make sense. Probably I'd favor the one by BF.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.