1
$\begingroup$

I ran a within-subjects mixed model using lmer() and the binary fixed effect came out significant. However, when I use the r.squaredGLMM on the model, the R2m is 0.001. Should I interpret the fixed effect as having no effect given the low R2m?

Here's the MLM

lmeModel = lmer(y ~ x + (1|PID)

Here's the model output:

Random effects:
 Groups   Name        Variance Std.Dev.

 PID      (Intercept)  3.609   1.900   
 Residual             10.703   3.272   
Number of obs: 6645, groups:  PID, 443


Fixed effects:
               Estimate Std. Error         df t value             Pr(>|t|)    

(Intercept)    6.46135    0.10425  539.50609  61.980 < 0.0000000000000002 ***

x              0.43405    0.09114 6560.98002   4.762           0.00000195 ***
$\endgroup$
2
  • $\begingroup$ What happens to the estimate for x when you add further covariates to account for selection factors? It may also be worth considering interactions between x and covariates. $\endgroup$
    – Erik Ruzek
    Commented Apr 28, 2021 at 21:51
  • $\begingroup$ The data is from an experiment so I don't have many more covariates to add. I did try the model including random slopes for X and for the trial number, but that didn't change anything much. $\endgroup$
    – Benji
    Commented Apr 29, 2021 at 22:17

1 Answer 1

0
$\begingroup$

It would be good to double check that the $R^2$ looks sensible; we could do this if we know the standard deviation of $x$. Is SD(x) around 0.3ish? (If not, I might have this wrong!).

What is a big $R^2$ of course depends on your application, but it does sound rather like a situation of, tiny effect, but a lot of data to prove it. Significance only asks if you have enough data to suggest that $x$ might not have no effect. You have 6645 data points. Although they are drawn from only 443 PIDs, it's clear from the output that most of the variance is from one observation to the next, rather than from one PID to the next. So you still have a lot of fresh information in each of those 6645 data points. And with a lot of data, you can get a significant result for even a tiny effect. Because that's still enough evidence to start to look like it's not no effect.

So in summary, you're pretty confident that $x$ doesn't have no effect (from the significance). But you're also pretty confident it has doesn't have much of an effect at all (that is clear from the small $x$ coeff with even smaller SE).

(Not sure if there should be caveats about whether the significance testing maths breaks down here or not).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.