In a survival analysis, I would like to prove that a variable Y is more associated with an outcome than another variable Y'. My goal is only to prove better association, not to use any model for prediction.

For instance, let's take 2 models, adjusted for known confounding factors:

m1 = coxph(Surv(start, stop, event) ~ Y' + X1 + X2 + X3, data=db)
m2 = coxph(Surv(start, stop, event) ~ Y + X1 + X2 + X3, data=db)

The Surv() object is some clinical outcome (cancer, diabetes, etc.) and Y' is the update of Y, which is an assessment score. For instance, you could consider Y as the BMI and Y' as weight/height^3, or as Trefethen's "new BMI".

My hypothesis is that Y' "explains more" the outcome than Y, and indeed, HR are far broader for Y'.

But since m1 and m2 are not nested, I am not aware of any test to compare them directly.

I heard about Harell's C statistic, but according to some ressources, it "measures of the ordinal predictive power of a model", and "it should not be taken seriously if it is calculated in the dataset in which the model was fit".

Then, is it possible to test the comparison of Y and Y' ? Is there any measure or coefficient of the superiority of one over the other ?


On @EdM advices, I computed a model with both Y and Y' and did some LRT and extracted the AIC. Here are the results:

m12 = coxph(Surv(start, stop, event) ~ Y' + Y + X1 + X2 + X3, data=db)  
# [1] 57421.23

# [1] 57461.42  
#  loglik  Chisq Df P(>|Chi|)    
#1 -28708                     
#2 -28727 36.952  1 1.842e-07 ***   

# [1] 57692.19
#  loglik  Chisq Df P(>|Chi|)    
#1 -28708                        
#2 -28842 267.72  1 < 2.2e-16 ***

Harrell's answer to a similar question on this page suggests that you might first consider whether such a comparison is really useful.

If both Y and Y' will always be available for the same cases and your interest is in prediction, then a model including both of them might well be superior to models limited to only one of them. If one is more difficult or expensive in some way, you could proceed by seeing if adding the more difficult/expensive measure to a model already containing the less difficult/expensive measure improves the fit, with standard log (partial) likelihood tests, as you would now be comparing nested models.

Some hold that AIC can be used to compare non-nested Cox models, but so far as I know this would simply be a heuristic estimate of the "better" model with no rigorous statistical test. Some even question whether AIC should be used for comparing non-nested models; see this page for discussion.

The generally perplexing issue of predictor selection is even more difficult for Cox regression, in which (unlike in linear regression) omitted-variable bias can result if any outcome-related variable is omitted from the model, even if it is uncorrelated to any included variable. This problem has long been appreciated. So if both Y and Y' are likely to be related to outcome, consider including them both or judging whether the more difficult/costly measure adds anything to the other.

If by "Y' is the update of Y" you mean an update at a later time during the study (rather than a potentially improved version of the same basic test, as I implicitly assumed above), then you are getting into issues of time-varying covariates. In that situation, you presumably have initial values of Y for all cases and the value of Y' becomes available later in time. Given the general principle that it's unwise to omit potentially useful data, the best test would be to include the original Y value in all models and see whether the update-in-time of Y' adds anything by comparing the appropriate nested models.

  • $\begingroup$ Thanks for you answer, as usual. You made me realize I should have explained better what I meant by "update". Please take a look at my question again, I think you will agree that keeping both in a final model makes little sens. And since Y and Y' are highly correlated, can I really perform a LRT to see how much one add to the other ? $\endgroup$ – Dan Chaltiel Aug 24 '18 at 13:01
  • $\begingroup$ @DanChaltiel the LRT would test whether adding one to the other improves the model. With correlation, it's possible you would find that Y' doesn't significantly add anything to Y, and also vice versa. See this page for discussion and links on testing non-nested models. You could try comparing predictions between the two competing models with bootstrapping. If you are dealing with a composite variable like BMI, consider looking at each of its components; see this page for suggestions. $\endgroup$ – EdM Aug 25 '18 at 14:08
  • 1
    $\begingroup$ +1 Good try at responding constructively to a confusing, changing question. $\endgroup$ – whuber Aug 27 '18 at 11:33
  • $\begingroup$ @EdM I edited my question so you can see some results. To me, it seems that removing Y or Y' remove a lot of information from m12, but I don't think I can compare the two and I am not sure AIC can change this. Though it seems that the anova removing the updated score Y' have a lower p-value, but does this mean I am removing more information ? (I am never sure what I am allowed to think based on a p-value). $\endgroup$ – Dan Chaltiel Aug 27 '18 at 12:17
  • $\begingroup$ @whuber total agree on this (I won't set it as accepted because for now it doesn't solve my question). On the other hand, could you please elaborate on why this question is confusing ? It would help a lot more than just stating it. It is very difficult to write a clear question when you are still a beginner. For the "changing", I disagree, I don't think I deliberately changed the purpose of my question (except for the "update" explanation). $\endgroup$ – Dan Chaltiel Aug 27 '18 at 12:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.