I've fit a stratified Cox proportional hazards model to some survival data, where I've stratified by a potential confounder which is the batch the data comes from (there are three batches).

Now, I'd like to externally validate this model on a new set of data, that also come from the three batches (independent samples though). Should I be using the strata information in this prediction? My thoughts are to exclude the strata, as I want to remove this confounding factor and not artificially inflate my predictive value (C-index etc).

Is this correct?

Edit for clarification:

Say I'm developing a diagnostic test for some disease, using a stratified Cox model to account for a confounder (a batch effect like which hospital the data was collected at). Now, if I go to apply this test in the real world, a new individual won't have the batch effect associated with my study, so there's no point in assigning them to some pre-defined stratum.

  • $\begingroup$ You should still fit a stratified model. The model in training and validation part should remain the same. $\endgroup$ – Peter Oct 16 '14 at 12:21
  • $\begingroup$ The main issue is the validation: I can validate C-index within each stratum separately, or I can evaluate over all individuals ignoring the strata, which leads to different results. $\endgroup$ – purple51 Oct 20 '14 at 5:01

As I understand your question you have two issues. First, you are stratifying on a factor that has levels that may not exist in future datasets for which you want predictions. This is a sticky issue, even more so in stratified Cox models because you can't even easily average over strata effects as you can for fixed additive effects in a non-stratified Cox model. One possible approach is to get predicted survival curves for all strata and to do a time point by time point averaging of the multiple step functions. It is worth considering whether you expect the strata to engender enough non-proportional hazards to be worth handling them as strata instead of covariates.

Your second issue is how to do the validation when a variable is not part of the covariate portion of the model. In the R rms package val.surv and calibrate functions this is done by getting calibration curves for survival probabilitiies at a specific time point. This takes strata into account and allows the stratification variable to contribution to predictive discrimination. If you want to not get any credit for predictive discrimination for having the strata then you can take the linear predictor $X_{test}\hat{\beta}_{train}$ and fit it as a single variable in the test sample, stratified by the new stratification factors, and judge how close to 1.0 is the coefficient of the linear predictor.

  • 1
    $\begingroup$ Yes, you understood perfectly. I'm dealing with weak predictors where just knowing the strata already gives a C-index of 0.6, so I'm trying hard to remove the strata effect. $\endgroup$ – purple51 Oct 24 '14 at 1:41
  • $\begingroup$ Just be sure that you don't want to claim credit for the information available from the strata. $\endgroup$ – Frank Harrell Oct 24 '14 at 2:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.