# Comparing non-nested Cox models: AIC & Partial likelihood test?

I'm doing survival Cox PH analyses and want to understand how an important continuous variable impacts time-to-event (model1).

This important variable is frequently dichotomized in my research field (model2). I'm studying how a new cutpoint (model3) would perform.

When building a Cox model, anova(modABC, modABCD, test = 'LRT') is pretty straightforward to use when comparing nested models.

But what about non-nested Cox models? If I would like to see how continuous (model1) compares to conventional binary (model2) or to new cutpoint (model3)?

AIC seems like the most logical, as common pratice and shown here.

But for instance, is merely having a smaller AIC value sufficient for stateing - considering proper power (n of events), careful assumptions checking - model3 seems a better choice than model2?

> extractAIC(model1)
[1]    8.000 2635.941
> extractAIC(model2)
[1]    8.000 2640.818
> extractAIC(model3)
[1]    8.000 2638.635


I checked this post, pointing to partial likelihood tests, from this 2002 Biometrika paper.

With nonnestcox package it can be implemented:

> plrtest(model3,model2, nested = FALSE)

Variance test
H0: Model 1 and Model 2 are indistinguishable
H1: Model 1 and Model 2 are distinguishable
Fine: p = 8.05e-05
Non-nested likelihood ratio test
H0: Model fits are equally close to true Model
H1A: Model 1 fits better than Model 2
z = -0.265,   p = 0.605
H1B: Model 2 fits better than Model 1
z = -0.265,   p = 0.3953
H1: Model fits not equally close to true Model
z = -0.265,   two-sided p = 0.7907


Is this an appropriate way to carry the analysis? I also have trouble understanding Fine: p = 8.05e-05, indicating distinguishable models, but "high" p-values for H1A and H1B. Would H1B be a better fit? I mean, I know variance is a different test than partial likelihood.

Thanks a lot! I'm a new R user. Also running other regression modelings, but this Cox part is important to the big picture. Also know dichotomous approach for continuous variable is not stats gold standard.

• I'm also solving the same question and I'm a bit stuck with why variance test is so significative but none model seem to follow the H1 in the likelihood test... what's the interpretation of this? Is this saying that any model is relevant? Dec 15, 2021 at 15:27
• I faced this issue for a while and couldn't make much sense of this. I ended up moving from this approach. Without showing a model test p-value, results were already pretty clear to the point of the story (three bins better than two, as a starting discussion towards more continuous approaches). Dec 17, 2021 at 10:01

Convincing a reader that you can use the AIC to choose among non-nested models depends on the reader's appreciating what AIC means.* Someone with that level of statistical sophistication will probably already be convinced that a search for a "better" cutoff of a continuous predictor is not worth the effort. I'm struck that even the simple linear form for the continuous predictor used for model1 seems to outperform both cutoffs of model2 and model3 in terms of the AIC.
With respect to the Fine test, I don't see a discrepancy in the results (although I'm not an expert on that). The Fine test tries to determine which of a set of models might be closest to a "true" model. As I understand it, you can tell that your model2 and model3 differ from each other (the first reported test), but the Fine test can't determine that either of them is closer to the "true" model than the other (the second set of tests). Maybe they are in some sense on different "sides" of the "true" model yet equally far away from it.