Am I dealing with within-cluster dependence or independence? (Cox proportional hazards model/frailty model)

Question

this is my first time asking a stats questions anywhere online. I'm a young female PhD student and quite anxious about this, so if possible, please be kind. I'll do my best to ask my questions clearly.

Background: For my project, I'm running a Cox proportional hazards model with a binary outcome, two fixed effects (one also binary, one continuous) and one random effect. From my understanding, this is also called a "frailty model" (because it has only one random effect). I fit it in R using the "coxph"-command (and it seems like "coxme" and "frailtyPenal" are valid alternatives, as they produce similar results; however, I plan to report results based on "coxph").

The random effect is subject. Individual test subjects were assessed across three trials, producing three values for the binary outcome per subject.

Now I want to assess the impact of the combined predictors via a full vs. null model comparison. Typically I would construct a null model in R (with outcome ~ 1 or, for mixed models as this, outcome ~ (1|ID)/cluster(ID)) and compare my full and my null model with a LRT using the anova command (with test="Chisq"). It turns out constructing null models the way I normally do in R does not work for this model. From my understanding, since the clusters adjust the fixed effects, the model cannot be created if the clusters have nothing to adjust. (I hope that makes sense?)

Anyway, my solution is to simply use the summary output after constructing the model and checking the significance of the combined predictors by looking at the omnibus tests.

QUESTION: Which omnibus test is most appropriate in my case? LRT, Wald test, Score (logrank) test or robust score test (these are the ones the "summary"-command outputs)? Can I assume "independence of observations within a cluster"? (If not, it seems I should use the Wald or robust score test, right?)

My own thoughts/the part that confuses me: On the one hand, it seems logical that there is dependence within the clusters. One cluster means one participant. It seems obvious that there would be error variables that affect each participant (and thus cluster) in the same way across trials. However, on the other hand, in our design we did what we could to ensure as much independence as possible between the three trials. Participants were free to choose their behavior during each trial, regardless of the other trials. Some showed the behavior of interest on all trials, some only on the first or second or third (or any two). Some never showed it. So in this sense, perhaps the trials could be argued to be independent? And in that case, I could rely on the LRT, right? The p-values for the LRT are much smaller than for the other tests, but I do not know whether I can/should base any conclusions on them.

TLDR: I've tested subjects across multiple trials. These are "clusters" in my survival analysis/Cox proportional hazards model. Behaviors were freely chosen on each trial, yet by the same individuals. Is this a case of within-cluster independence or dependence?

Thank you so much in advance for your help. I really appreciate it. And I hope this was understandable.

Additional information in response to a helpful comment:

• In the three trials, we modelled both whether the outcome occurred and when it occurred (in seconds). Hence why we opted for a survival analysis.
• The main difference between the three trials was their order in time. However, as this was a psychological study meant to simulate natural interactions, there were also minimal differences in phrasing (e.g., adding the word "again" to a sentence to acknowledge the repetition and make the situation seem less strange, but otherwise repeating it verbatim).
• The choice of the behavior of interest (opting to do it or not do it) is the binary outcome. As mentioned, if people opted to do it, we also assessed how quickly they did it.

Answer 1

Welcome to Cross Validated, Sophie :)

Can I assume "independence of observations within a cluster"?

If I have understood the details in your question then the answer here is No. I appreciate that you took steps to try to eliminate dependency:

in our design we did what we could to ensure as much independence as possible between the three trials

Nevertheless, the starting point should be to assume non-independence. Your clusters are individual subjects, and typically, when we make repeated measures on the same subject, we expect these to be more similar to each other than measures on another subject. So when you fit your model, you may find that this dependency is so small that you can ignore it, but you won't be able to know this in advance.

Which omnibus test is most appropriate in my case? LRT, Wald test, Score (logrank) test or robust score test (these are the ones the "summary"-command outputs)? Can I assume "independence of observations within a cluster"? (If not, it seems I should use the Wald or robust score test, right?)

A few points on this:

• Wald test: this is not an omnibus test. It is used for evaluating individual parameter estimates rather than assessing global model fit.
• The log-rank test: this does not extend to Cox models with random intercepts (I would recommend reading Frank Harrell's work on this )
• The robust score test: Unfortunately I do not know much about this, but sounds promising for your use case.

For assessing the global fit of a Cox proportional hazards frailty model with a single random effect for subjects, the Likelihood Ratio Test (LRT) is a common choice when comparing nested models, such as a full frailty model with all predictors and the random effect, against a reduced model with fewer predictors or without the random effect. The LRT evaluates the overall significance of the predictors and the random effect in explaining the outcome.

Note that the term "nested" is used here in the sense of a reduced model being nested within a more complex model - it has nothing to do with groups being nested (ie., nested random effects). As Frank explains, it can be sensitive to model misspecification which arises when within-cluster dependence isn't fully accounted for, potentially inflating the significance of the test.

It is also important that when comparing nested models:

• Comparing models that are fitted with Restricted Maximum Likelihood (REML) and differ in their fixed effects never makes sense.

• Using AIC/BIC/p-values to compare the same model fitted with REML vs ML never makes sense; you need to make the decision which method to use on a priori, theoretical grounds.

• Comparing models that are fitted with REML and differ in their random effects is justified.

Note I have taken those bullet points directly from Ben Bolker's answer here:

Citation for ML vs. REML