4
$\begingroup$

I have a medical background, with absolutely no knowledge about programming. I am working on survival analysis. I have fitted a multivariate Cox model which I need to cross-validate. My questions are:

  1. My dataset is small (79 samples & 37 events). Which is the better method of cross-validation - bootstrapping or leave-one-out method?
  2. I am using the rms package. I need someone to explain to me in plain English how to enter the code. I have tried reading the help on validation, calibration, but I have not understood much. I tried validation by various methods.

I have indicated below the error message I get. Can someone please explain to me what is going wrong and also give me the correct code to be used for: bootstrapping and cross-validation by leave-one-out method.

I have a cox model with four variables: v1, v2, v3, v4. v1 is a factor with three levels; v2, v3 and v4 are factors with two levels.

data=Model5
fitmodel5 <- cph(Surv(time,event)~v1+v2+v3+v4,na.action=na.omit,data=Model5,x=TRUE,y=TRUE,surv=TRUE)

 val1 <- validate(fitmodel5, method="crossvalidation", B = 10, dxy=TRUE)**
Error in measure(xb[test], y[test, , drop = FALSE], strata = stra[test],  : 
  fit failure in discrim,coxphFit
In addition: Warning message:
In fitter(..., strata = strata, rownames = rownames, offset = offset,  :
Ran out of iterations and did not converge

 val1 <- validate(fitmodel5, method=".6", B = 100, dxy=TRUE)
Weights for .632 method (ordinary bootstrap weights 0.01)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
0.00768 0.00925 0.01010 0.01000 0.01070 0.01290 
Divergence or singularity in 3 samples
Warning messages:
1: In fitter(..., strata = strata, rownames = rownames, offset = offset,  :
  Ran out of iterations and did not converge
2: In fitter(..., strata = strata, rownames = rownames, offset = offset,  :
  Ran out of iterations and did not converge

 val1 <- validate(fitmodel5, method="boot", B = 100, dxy=TRUE)
Divergence or singularity in 1 samples
Warning message:
In fitter(..., strata = strata, rownames = rownames, offset = offset,  :
Ran out of iterations and did not converge

Thanks

$\endgroup$
0

1 Answer 1

4
$\begingroup$

To do this well takes not so much a programming background but significant training in multivariable modeling. There are many aspects of the problem that need to be addressed first, including the causal pathway, appropriateness of predictors in other ways, appropriateness of the outcome variable, reliability of measurements, variable redundancy and data reduction, relaxing the linearity assumption, dealing with the additivity and proportional hazards assumptions, overfitting, mastering indexes of predictive accuracy, model validation, model interpretation. [I'm the author of the R rms package]

$\endgroup$
3
  • 1
    $\begingroup$ When I read the question, I checked ?cph and I knew I already heard about the author... Now I know where I heard about you ;-) $\endgroup$
    – ocram
    Jan 5, 2013 at 19:18
  • $\begingroup$ I'm having the same problem... it's strange because with few repetitions (B = 100) it works well but not with more (B = 200), and it prompts the same error. Any way to understanding what's happening? Where can I read something about correctly build models? Thanks $\endgroup$
    – jgarces
    Jan 20, 2022 at 15:57
  • $\begingroup$ Without knowing more about the data I'd suggest trying B=300 preceded by set.seed(i) for various i until it gets through all 300. $\endgroup$ Jan 21, 2022 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.