6
$\begingroup$

I'm studying several schemes on classifying patients about their survival time. Let me illustrate the problem with supposing I have just two schemes.

Let's suppose that Scheme 1 put the patients in 5 groups, and Scheme 2 put the patients in 5 groups too (but the groups have different compositions, for example patient 1 in Scheme 1 may be in the group 3 and in Scheme 2 in the group 4).

So, it is my problem but with several schemes.

KM plots are useful to analysis, but it is not practical, and some plots are very similar in visual terms. The logrank test is not the case I think, because I think it is just useful to verify if I really have any difference in the survival experience between the curves and no more.

In really, what I want is to verify which scheme give me a better prediction in some sense of accuracy, in really a measure for that. One way I think was to just use the survivalROC package and apply the survivalROC function for times 60, 120 and 180 for example, and what which scheme give the best AUC.

Another way I thought is to use these schemes with cox proportional hazard models, is it really an alternative? Using the coxph function I can have $R^2$ and concordance in its return, and I can compute the AIC with the extractAIC function too and choose the best scheme using the less AIC.

In the AIC case I have some doubts beacuse I thought that AIC was good a time ago, but when was I reading recently about it I just found examples with nested models for COX PH, is it applicable to non-nested models too?

So, in summary I thought about 4 options:

  1. survivalROC and use AUC
  2. coxph and use $R^2$
  3. coxph and use concordance
  4. coxph and use extractAIC

I want to automate my analysis using one numerical measure to automatic selection of the best scheme (remembering the non-nested models nature).

Is there any problem in this approach I am thinking? Which could be the best measure for that?

|cite|improve this question
$\endgroup$
1
$\begingroup$

I figure the schemas' groups does not come with an absolute probability of an event in any given period. In that case, you care about the ranking which leads to focusing on AUC or concordance of the options you have given.

The concordance measure is a generalization of AUC to time outcomes. So unless you really care about a specific point in time then I would say you use the concordance measure rather than an AUC at a fixed point in time.

You can get a concordance measures for either of your model with survival::survConcordance after having changed you schema outcome into numeric variable with the given ordinal scale and setting them on the right hand side of the ~ in the formula argument.

Caveat

My assumption is that you are not doing the modelling but are given models/schemas which you have to compare where each model/schema gives you ordinal grouping of risk for each person. However, if you do the modelling then I agree with @Frank Harrell comment

Reject the ordinal groups. Start over. Inappropriate.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ The problem is not well conceptualized, the ultimate goals not stated, and the processes suggested involve a great deal of model uncertainty that will destroy much of the meaning of the final result. The very idea of 'classifying' patients is appalling. Continuous predictions (e.g., predict an entire patient survival curve given specific covariates) is more what is needed. $\endgroup$ – Frank Harrell Oct 14 '17 at 13:13
  • $\begingroup$ I agree but it seems like the OP has been given two models/schemas that produces ordinal groups (I assume) and now you want to evaluate the two against each other. $\endgroup$ – Benjamin Christoffersen Oct 14 '17 at 13:30
  • 1
    $\begingroup$ Reject the ordinal groups. Start over. Inappropriate. $\endgroup$ – Frank Harrell Oct 14 '17 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.