# How to interpret the Wald Chi-square test for the frailty term in a mixed-effects Cox survival model?

Background

I'm fitting a Cox model to assess the relationship between treatment (binary) and time-to-event (event is also binary) while also controlling for 3 or 4 covariates. Because I want to account for the fact that individuals can have multiple events, I've added a frailty term to the model at the level of an individual (their ID variable, to be precise; see e.g. Therneau & Grambsch, 2000, and Balan & Putter, 2020). To do all this, I'm using Terry Therneau's survival package and coxph function in R, and within that, the frailty argument.

I've gotten the thing running well and it's yielded a result. Running summary() on the model yields this:

                        coef       se(coef)   se2          Chisq    DF   p
treatment               0.2637760  3.908e-02  2.856e-02     82.27    1   1.2e-19
covariate_1             -0.04549   3.002e-02  1.209e-02      0.70    1   4.0e-01
covariate_2             -0.04195   6.181e-03  2.063e-03    133.77    1   6.1e-31
covariate_3             -0.28931   4.688e-02  1.592e-02     32.22    1   1.4e-08
frailty(ID)                                              87186.90 4197   0.0e+00


The rest of the output exponentiates the coefficients, giving me nice, interpretable Hazard Ratios and CI's. All well and good. (I've futzed with the numbers here for privacy's sake, but I've kept the gist of the truth.)

The Problem

However, I'm not sure what exactly to make of the Wald test for frailty(ID) and its p-value. The Therneau and Grambsch text's relevant section doesn't really speak to its interpretation, and the book's vignettes feature both statistically significant and non-significant Wald tests for the frailty terms they're guiding the reader through, leading me to wonder about their practical meaning. (If the authors aren't pointing out the implications for the frailty term's significance in their vignettes, how much could it actually matter in real-world science?) Here's a relevant snippet, referring to R output they've just cited:

An approximate Wald test for the frailty is 17.7 on 14.4 degrees of freedom. [...] A likelihood ratio test for the frailty is twice the difference between the log partial-likelihood with the frailty terms integrated out, shown as "I-likelihood" in the printout, and the loglikelihood of a no-frailty model, or 2(181. 7 - 180.8) = 1.8. It has one degree of freedom and p-value = 0.18, similar to the Wald test. (pp. 235-236)

It's important to note that I'm not a statistician (though you know that by now, probably), but still, I can't make much of this lone reference to these tests. My model's "I-likelihood" result is I-likelihood = -352655.3, and my LR test result for the model is Likelihood ratio test= 88877 on 4211 df, p=<2e-16, but I'm not sure how these relate to the frailty term, nor how to interpret them in light of the frailty term.

My best guess, going on a sort of "skeleton" of intuition about other, simpler tests I've had in my applied statistics education, is that the Wald test for frailty has a null hypothesis that's assuming something like "there's no frailty to speak of here", and my significant result is a suggestion to reject that H0. But again, I am not confident here.

What do I make of that frailty(ID) and its p-value?

Yes, your sense that the "null hypothesis that's assuming something like 'there's no frailty to speak of here', and my significant result is a suggestion to reject that H0" is correct. You can see this on page 237 of Therneau and Grambsch, where they evaluate a profile likelihood to get confidence intervals for the frailty term instead of the Wald test: "The 95% confidence interval for $$\theta$$ is (0, 1.8); there is no clear evidence for a frailty effect."

With this large a data set it's likely that all coefficients will be "statistically significant." Your major task is to evaluate the practical significance.

As Therneau and Grambsch note on page 236: "Formal justification for the approximate Wald test of the frailty term is lacking..." To use the alternative likelihood-ratio test, you would have to follow the calculations that you quote by fitting the model again without the frailty term to get "the loglikelihood of a no-frailty model" to use together with the "I-likelihood" value that you have from the full model.

They show on pages 236-237 how to calculate a profile-likelihood evaluation, based on refitting the model over a range of specified values for the frailty coefficient. Profile likelihood evaluations are usually more reliable than Wald tests. Plotting the likelihood profile will generally give better estimates of confidence intervals for the frailty coefficient, although with this size data set the asymptotic equivalence of the Wald and the likelihood-ratio test might lead to similar estimates.

• Thanks very much for your considered and clear reply, EdM. One question: I get the arithmetic you laid out, but on p. 137 right after they do it (2(181. 7 - 180.8) = 1.8) they say "it has one degree of freedom and p-value = 0.18". How did they get that p-value? Commented May 28, 2022 at 17:41
• @logjammin that's the probability that a chi-square distribution with 1 degree of freedom has a value of 1.8 or greater. In R: pchisq(1.8,1,lower.tail=FALSE) gives a probability of 0.1797. Most of these tests end up being based on chi-square distributions.
– EdM
Commented May 28, 2022 at 17:44
• Amazing, cheers. I'm going to run the no-frailty model a little later and do the math. This'll really bolster my interpretation of the results. Can't thank you enough again for this and all the great work you do around here. Commented May 28, 2022 at 17:45
• @logjammin doing the full likelihood profile is even a better way to bolster your interpretation of the results. That shows how the likelihood varies as a function of the frailty coefficient.
– EdM
Commented May 28, 2022 at 17:47