Looking for ways to compare between coxph models

Question

I'm running Cox proportional hazards regression in R, and would like to test the option of categorizing one of my continuous variables to factor (I'm aware of the loss of data issue, just checking).

Another thing that I'd like to check is putting the difference between 2 cont. variables inside the regression instead of putting them both. [It's actually testing if the pulse pressure, e.g, the difference between the systolic and diastolic blood pressure is more significant than each of them separately]

My question is: What is the best way to compare between the different variations of the regressions (lets assume that I'll use step() in each of the attempts). No missing values in the dataframe whatsoever.

I'm pretty confused between AIC, R2 (of coxph) and concordance of coxph. Can anyone clear things up for me? Is there any other option of comparing between different models on the same data?

Thanks!

Answer 1

I certainly wouldn't use step() here (actually, I would strongly prefer not to use it anywhere). Here I would force entry of all the variables.

Then I would use predict to get predicted values from each model and compare them to the actual values, probably through a scatter plot. I might also look at the differences between the predicted and actual values, see the range for each model, perhaps make parallel box plots, then maybe a t-test or something like that.

Answer 2

I would use the second-order Akaike’s Information Criterion ($AIC_c$) for model selection, which is $AIC$ corrected for small sample size using a bias-correction term. $$ AIC = -2ln(L)-2k $$ $$ AIC_c = AIC+\frac{2k(k+1)}{(n-k-1)} $$ where $ln(L)$ is the log-likelihood, $k$ is the number of parameters, and $n$ is the sample size. Sometimes $AIC$ can perform poorly when the ratio of sample size to the number parameters in the model is small (Burnham and Anderson 2002).

$AIC$ or $AIC_c$ can be recaled to $\mathsf{\Delta}_i=AIC_i-minAIC$ where the best model will have $\mathsf{\Delta}_i=0$. Further, these values can be used to estimate relative strength of evidence ($w_i$) for the alternative models where: $$ w_i = \frac{e^{(-0.5\mathsf{\Delta}_i)}}{\sum_{r=1}^Re^{(-0.5\mathsf{\Delta}_i)}} $$ This is often refered to as the "weight of evidence" for model $i$ from the model set. As $\mathsf{\Delta}_i$ increases, $w_i$ decreases suggesting model $i$ is less plausible. Also, the weights of evidence for the models in a model set can be use in model averaging and multi-model inference.

References:

Burnham, K. P., and D. R. Anderson. 2002. Model selection and multimodel inference: a practical information-theoretic approach. Second edition. Springer, New York, USA.

Anderson, D. R. 2008. Model based inference in the life sciences: a primer on evidence. Springer, New York, USA.