2
$\begingroup$

I have two linear mixed models that I ran with the lmer function from the lme4 package. The models are identical except that I calculated the input fixed factor in slightly different ways. I now want to test whether the fixed effects estimates output by the models significantly differ.

What is the appropriate test for doing this?

In one model, the fixed effect estimate is -9.1. In the other, the fixed effect estimate is -2.0. So, qualitatively, they seem significantly different, although I do not know how to make this judgement statistically.


A longer description of my problem follows here...

I have these two models:

M1 <- lmer(result ~ IQ + (1|Participant), data = DF)

M2 <- lmer(result ~ IQ + (1|Participant), data = DF)

As you can see, they are identical -- I'm trying to test whether participants' (my random factor) result vary by their "intelligence", or IQ, which is a binary categorical variable such that participants are either classed as high or low. However, IQ was evaluated with a different test in M1 and M2. If it helps, you can think of is as "analytic IQ" in M1 and "creative IQ" in M2.

The output of M1 displays a fixed effect estimate of -9.1 (i.e., roughly, a low IQ individual scored 9.1 points worse than a high IQ individual).

The output of M2 displays a fixed effect estimate of -2.0.

Qualitatively, it appears that the IQ test used in the case of M1 ends up categorising participants in a way that is significantly different than the IQ test used in M2. But my question is: how might I compare the fixed effect estimates to make this judgement, statistically?

$\endgroup$
  • 1
    $\begingroup$ Can you elaborate on what you mean by 'slightly different ways'? $\endgroup$ – Jeremy Miles Sep 26 '19 at 20:56
  • $\begingroup$ Please see my edits to the original post. Hope it helps? $\endgroup$ – user72716 Sep 27 '19 at 10:41
  • 1
    $\begingroup$ Suggestion: please call your variables different names. The analytic IQ is a different variable than the creative IQ. $\endgroup$ – Dave Sep 27 '19 at 10:57
3
$\begingroup$

As far as I can see the models are not nested in this case and therefore you cannot perform one of the classic statistical tests (i.e., likelihood ratio test, score test, Wald test).

You could compare the AIC/BIC values (which in this case should be equivalent to only looking at the log-likelihood values), and see which one is smaller.

$\endgroup$
  • $\begingroup$ Would there be anything wrong with checking for overlapping confidence intervals of the parameters? If creative IQ is (-3,-1) and analytic IQ is (-10,-8), that would tell me that they're different. Or does the lack of nesting ruin this plan? $\endgroup$ – Dave Sep 27 '19 at 10:56
  • $\begingroup$ No, that won’t tell you if one or the other is better. $\endgroup$ – Dimitris Rizopoulos Sep 27 '19 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.