I have a logistic regression model of the structure y ~ x1 + x2, and a generalized linear mixed model (GLMM) with random intercept and random slope, of the structure y ~ x1 + x2 + (1 + x2 | id). My goal is to determine whether a mixed model is necessary. Are these two models considered "nested", and can they be compared with a likelihood-ratio test (LRT)? How would I determine the degrees of freedom of this test?

If they can be compared with a LRT, how does one mathematically state the null hypothesis? Say $G = cov(b_i)$, where $b_i$ is the $2 \times 1$ random effect vector for the GLMM model. Would the appropriate null hypothesis be $H_0: G = 0$?

If the above setup for the hypothesis is correct, how would the distribution of the resulting test statistic be identified. I am aware that testing $H_0: g_{22}=0$ requires the distribution of the test statistic, under the null, to be a 50:50 mixture of $\chi^2_1$ and $\chi^2_2$ because the null hypothesis is on the boundary of the parameter space. But in my case, I want to test whether the entire matrix $G$, which is $2 \times 2$, is zero.


1 Answer 1

  • Yes, they are nested: the mixed model reduces to the simpler model if $\sigma^2_1=\sigma^2_{x_2}=0$. (This is the same as $G=0$, because the covariances must be zero if the variances are, but stating it in terms of a joint condition on $\{\sigma^2_1, \sigma^2_{x_2}\}$ is probably easier to understand.)
  • The likelihood ratio test in its usual form doesn't work right — it's conservative — because the derivation of the likelihood ratio test depends on a Taylor expansion of the log-likelihood around the null parameters, which doesn't work if the null parameters are on the boundary of the feasible model space (you can't expand around $\sigma^2=0$, because that implies that you're including negative variance values in your expansion). This is discussed in a variety of places (Self and Liang 1987; Stram and Lee 1994; Goldman and Whelan 2000; Pinheiro and Bates 2000). For simple models there is a known correction factor to the usual null distribution. For example if you're testing between models that differ by a single variance parameter (e.g. random intercept model vs. no-random-intercept model), the null distribution of $-2\Delta(\log L)$ is $0.5\chi^2_0 + 0.5\chi^2_1$, where $\chi^2_0$ is a point mass at zero; the bottom line here is that the nominal LRT p-value should be divided by 2. For more complicated models it's usually hard to derive, and people often calculate the p-value by parametric bootstrapping. The GLMM FAQ has a section on this ...

In particular, Stram and Lee (1994) discuss the geometry of some of the more complex cases (it's been a long time since I read it ...) The particular mixture of $\chi^2$s that forms the null distribution may be analytically derivable, but in my experience people usually give up and find the null distribution by simulation. The example below is from Pinheiro and Bates (2000) p. 87 (via Google Books): they show computationally that the null distribution for a particular comparison (which would be 1|Worker vs. 1|Worker/Machine) is approximately $\sim 0.65 \chi^2_0 + 0.35 \chi^2_1$; they then more or less say that they go ahead and use the naive LRT because it's easier.

As shown in the above-linked GLMM FAQ section, you can use pbkrtest::PBmodcomp() to get a valid p-value by parametric bootstrapping ...

enter image description here

Stram, Daniel O, and Jae Won Lee. “Variance Components Testing in the Longitudinal Fixed Effects Model.” Biometrics 50, no. 4 (1994): 1171–77.

  • 1
    $\begingroup$ thank you for your helpful answer. So a random effects model with slope and intercept term cannot be directly compared with the underlying logistic regression model? Instead, could perhaps the GLMM with random slope and intercept be compared to the GLMM with random intercept only, and followed by comparing the random intercept-only GLMM with the logistic regression model? $\endgroup$
    – bob
    Aug 7, 2020 at 0:02
  • 3
    $\begingroup$ It can be compared directly. It's just hard to know what the correct null distribution is. The problem applies when comparing the random-slopes model to the random-intercept model as well (because there we have the restriction that $\sigma^2_{x_2}=0$, which is again on the boundary). The section in the GLMM FAQ goes into a little bit more practical detail. $\endgroup$
    – Ben Bolker
    Aug 7, 2020 at 1:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.