# Cox PH model selection and validation

I am trying to analyze my data using survival CoX PH in SPSS v.19 and also attempting to make different prediction models (without and with a biomarker of interest). I am a clinician (not a biostatistician) and this is for a paper that I need for completing my Ph.D. For my model selection I used the following method. First, I identified the univariate predictors for the outcome, and then I used all of them in a Cox backward stepwise analysis to obtain my final model (I have only 25 events). Now, it was suggested to use some a priori variables (like for example age or sex - variables that weren't associated with the outcome in my univariate analysis and as such not included in the final model) and retain the variables only if they lead to an improved model fit (as assessed by LLR).

My first questions are:

1. Is it statistically correct to use an a priori defined model and then add my statistically significant variables, comparing the LLR of the different models (using the hierarchical Cox analysis)?
2. If this is correct, how do I initially select a model to which I then add another variable?
3. How do I solve the problem of overfitting? Do I have to use bootstrapping for my final model?

I will also try to compare the final model with a model that includes my biomarker.

1. How do I save the predicted probabilities for each model in order to then compare them (I need them to perform C statistic and NRI analysis)? In SPSS in simple logistic regression there is this option. Is there any macro that could do this?
2. Is is correct to use the predictive probabilities from the simple logistic regression?

Your first method of bivariate screening and then backwards elimination is not recommended. These methods have numerous problems and have been extensively discussed here and elsewhere (search on "Stepwise" to find some).

Taking your questions one at a time:

Is it statistically correct to use an a priori defined model and then add my statistically significant variables, comparing the LLR of


the different models (using the hierarchical Cox analysis)?

This is not a bad method. It is certainly an improvement.

If this is correct, how do I initially select a model to which I then add another variable?


Based on theory, substantive knowledge, research questions etc.

How do I solve the problem of overfitting? Do I have to use bootstrapping for my final model?


If you have a lot of cases, then overfitting shouldn't be too much of an issue, but you can do things like use a training and test set if you are concerned.

Your latter questions seem to be more SPSS specific; I don't use SPSS.

• Thank you for your prompt response. So I have an a priori model with 5 variables, and from my analysis there are an additional four variables (the final model after backward stepwise). I perform a hierarchical Cox analysis for each additional variable and select the best model. At this point is it correct to make the same hierarchical analysis and include the remaining three variables in the analysis? – Dimitrie Apr 18 '14 at 12:36
• No, don't use backward stepwise at all. Start with your a priori model and add variables that change the parameters in important ways or that improve fit. That's your model. It may be that there are several that are good - you can compare them with AIC or BIC – Peter Flom - Reinstate Monica Apr 18 '14 at 12:40
• +1 for the stand against stepwise regression. @Dimitrie, see 'Modern Epidemiology' by Rothman, Greenland and Lash for an approachable discussion of potential variable selection techniques. – Fomite Apr 18 '14 at 18:03

I always like throw in a vote for penalized regression perspectives: using R's glmnet function you can fit a Cox model with all your variables, and it will automatically shrink the coefficient for non-important variables down to 0.

From the glmnet examples:

set.seed(1010)
n=1000;p=100
nzc=trunc(p/10)
x=matrix(rnorm(n*p),n,p)

beta=rnorm(nzc)
fx=x[,seq(nzc)]%*%beta/3
hx=exp(fx)
ty=rexp(n,hx)
tcens=rbinom(n=n,prob=.3,size=1)# censoring indicator
y=cbind(time=ty,status=1-tcens) # y=Surv(ty,1-tcens) with library(survival)
foldid=sample(rep(seq(10),length=n))
fit1_cv=cv.glmnet(x,y,family="cox",foldid=foldid)
plot(fit1_cv)
title("Cox Family",line=2.5)

• I know that R is better in many aspects than SPSS, but for the moment I can only work with SPSS. – Dimitrie Apr 18 '14 at 12:27