0
$\begingroup$

I used SPSS (v. 25) conducted a pair of nested multtlevel models (aka linear mixed models) where the outcome criterion is itself also nested within participants. I followed Singer and Willett's (2003) recommendations to include time as a level-1 (within-participant) predictor.

The "base" comparison model included no predictors except a fixed-effect (between-participant, level-2 factor) intercept and the random-effect time I mentioned just above.

The model I compared against that base model included those same two factors (i.e., intercept and time) and only added a fixed-effect, dummy-coded gender term.

The -2 log likelihood (-2 LL) for the base model was 28316. The -2 LL for the new extended model that added in gender was 27341. The deviance for this difference is 28316 - 27341 = 975. With 1 df, this difference is highly significant (critical χ2 = 5.02).

However, when reporting the inferential test for the gender term, SPSS indicates that the F-score for this term is 3.05, which is not significant (with dfs of 1 & 903.8 and α = .05).

In other words, adding the gender term made for a significantly better-fitting model. However, the term that was added that made for this better fit is itself not significant.

I am at a loss how to interpret this in practical terms. It seems that gender should be taken into consideration when understanding the outcome variable, but how do I reconcile explaining that gender itself is not significant?

$\endgroup$
1
$\begingroup$

First, think of this just for a linear model, without any random term.

Differences in deviance for a linear model can't just be compared to a $\chi^2$ reference distribution: the actual reference distribution is $\sigma \chi^2$, where $\sigma$ is the residual variance. If you knew $\sigma$, you could divide the deviance difference by it and compare to a $\chi^2$ distribution

The $F$ -test estimates $\sigma$ from the residuals, and divides the deviance difference by $\sigma$. To account for the information used up in estimating $\sigma$, you use an $F$ distribution -- but that doesn't matter much here since the .95 critical values for $\chi^2_1$ and $F^1_903.8$ are almost identical.

In your mixed model, things are slightly more complicated. The non-integer denominator df on the $F$ test are because the statistic doesn't have exactly an $F$ distribution. But the reason for the deviance difference and $F$-test to apparently disagree are the same

$\endgroup$
1
  • $\begingroup$ Indeed, dividing that difference in -2LLs by the level 1 (within participant) residual produces a "pseudo-chi-squared" close to that F-score. Cheers! $\endgroup$ – wes May 21 '20 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.