How to select the best classification scheme in survival analysis (SurvivalROC, R2, Concordance, AIC)?

Question

I'm studying several schemes on classifying patients about their survival time. Let me illustrate the problem with supposing I have just two schemes.

Let's suppose that Scheme 1 put the patients in 5 groups, and Scheme 2 put the patients in 5 groups too (but the groups have different compositions, for example patient 1 in Scheme 1 may be in the group 3 and in Scheme 2 in the group 4).

So, it is my problem but with several schemes.

KM plots are useful to analysis, but it is not practical, and some plots are very similar in visual terms. The logrank test is not the case I think, because I think it is just useful to verify if I really have any difference in the survival experience between the curves and no more.

In really, what I want is to verify which scheme give me a better prediction in some sense of accuracy, in really a measure for that. One way I think was to just use the survivalROC package and apply the survivalROC function for times 60, 120 and 180 for example, and what which scheme give the best AUC.

Another way I thought is to use these schemes with cox proportional hazard models, is it really an alternative? Using the coxph function I can have $R^2$ and concordance in its return, and I can compute the AIC with the extractAIC function too and choose the best scheme using the less AIC.

In the AIC case I have some doubts beacuse I thought that AIC was good a time ago, but when was I reading recently about it I just found examples with nested models for COX PH, is it applicable to non-nested models too?

So, in summary I thought about 4 options:

1. survivalROC and use AUC
2. coxph and use $R^2$
3. coxph and use concordance
4. coxph and use extractAIC

I want to automate my analysis using one numerical measure to automatic selection of the best scheme (remembering the non-nested models nature).

Is there any problem in this approach I am thinking? Which could be the best measure for that?

Answer 1

I figure the schemas' groups does not come with an absolute probability of an event in any given period. In that case, you care about the ranking which leads to focusing on AUC or concordance of the options you have given.

The concordance measure is a generalization of AUC to time outcomes. So unless you really care about a specific point in time then I would say you use the concordance measure rather than an AUC at a fixed point in time.

You can get a concordance measures for either of your model with survival::survConcordance after having changed you schema outcome into numeric variable with the given ordinal scale and setting them on the right hand side of the ~ in the formula argument.

Caveat

My assumption is that you are not doing the modelling but are given models/schemas which you have to compare where each model/schema gives you ordinal grouping of risk for each person. However, if you do the modelling then I agree with @Frank Harrell comment

Reject the ordinal groups. Start over. Inappropriate.