2
$\begingroup$

I have an existing AFT (accelerated failure time) model which I'm using on a new dataset, with the obvious intent of testing whether the model predicts the new data well.

A first step is to look at calibration. My understanding of this is to take the PI values predicted by the original model and use them as the only covariate in a new AFT model using the event times/censoring of the new data. The slope and intercept of this model will give me an idea of whether the model needs to be calibrated for the new data. Slope = 1 and intercept = 0 tell me no calibration necessary. Chances of that are quite slim, I'm guessing, so I need to calibrate the model for the new data which involves modifying the coefficients for the original PI calculation based on the calibration results. That's about as deep as I go and would appreciate some direction toward the mechanics of that process.

Once that is done and I have a calibrated model, I can make predictions on the new data and start looking at how well the model performs. There are a number of ways to do this - look at Kaplan-Meyer curves for defined subgroups of the data to see if actual matches predicted, compare actual risk and predicted risk for deciles (or some other size split) of the data. I'm not clear on what is the "right" test, and any advice is apprciated.

Improve this question
$\endgroup$
2
  • $\begingroup$ What's AFT model? $\endgroup$
    – Aksakal
    Jul 23, 2015 at 19:52
  • $\begingroup$ Accelerated Failure Time - essentially a parametric survival model based on Weibull distribution. I'll spell out the acronym in the original post. $\endgroup$
    – KirkD_CO
    Jul 23, 2015 at 19:54

1 Answer 1

Reset to default
4
$\begingroup$

AFT is wider than Weibull, including log-normal, log-logistic, and other families. If you really have external data that were not available for model fitting, see the R rms package val.surv function. There are many other functions in rms for rigorous internal validation.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Yes, I was speaking of my specific model and inadvertently made it sound like AFT was only Weibull. My mistake. Any references to the mechanics of those validation tools? $\endgroup$
    – KirkD_CO
    Jul 24, 2015 at 0:45
  • $\begingroup$ In the function's help files. $\endgroup$
    – Frank Harrell
    Jul 24, 2015 at 2:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.