4
$\begingroup$

I'm using a Cox proportional Hazards regression (R survival package) to model Credit card activation propension, ie, which people are more likely to make their first buy? To give more context: Defining target variable - Credit Card industry.

So I have:

birth: Credit card creation

death: Date of first buy

event: people use their card for the first time

Here's the model summary:

## Call:
## coxph(formula = Surv(TIME, EVENT) ~ IDADE_EMPRESA + ZERO_RATIO + 
##     AVG_VENDAS + UF_CE + UF_ES + UF_DF + VL_LIMITE_COMPRA_ORIGINAL + 
##     VL_LIMITE_PARCEIRO + SD_VENDAS, data = x)
## 
##   n= 32548, number of events= 1999 

## Concordance= 0.716  (se = 0.007 )
## Rsquare= 0.038   (max possible= 0.706 )
## Likelihood ratio test= 1252  on 9 df,   p=0
## Wald test            = 1326  on 9 df,   p=0
## Score (logrank) test = 1318  on 9 df,   p=0

What I have done so far: used 9 months of data to fit the model and 3 remaining months as a holdout validation set. Now, I'm not sure how to use the validation set, what I would like to do is the following:

  • Rank the clients who are more likely to buy within 30,60,90 days (ie, I don't want the the Survival estimation T > 30,60,90), then estimate AUC or Concordance index for each time period.

Is that even possible? What are the alternatives for reporting accuracy? I have checked http://dni-institute.in/blogs/cox-regression-interpret-result-and-predict/, but it seems they are doing the opposite of what I need.

NOTE: Survival analysis is new to me, but I'm well familiar with general ML concepts like Cross Validation, Overfitting and so on. Thanks!

EDIT1: I've found the survAUC package, but I'm not sure if i understood the parameters:

  train = get.data(is.train=TRUE)
  test =  get.data(is.train=FALSE)

  fit = fit.surv() # get coxph model

  surv.train = Surv(train$TIME, train$EVENT)
  surv.test = Surv(test$TIME, test$EVENT)
  lp = predict(fit, test)
  # returns 0.7270601 0.7272526 0.7274083
  AUC.cd(surv.train, surv.test, predict(fit), predict(fit, test), c(30, 60, 90))

EDIT2: Another option, survConcordance in the survival package:

  fit = fit.surv()
  test =  get.data(is.train=FALSE)
  surv.test = Surv(test$TIME, test$EVENT)

  survConcordance(surv.test ~ predict(fit, test), data = test)

 # Outputs
   n= 428 
   Concordance= 0.7799616 se= 0.03275571
   concordant discordant  tied.risk  tied.time   std(c-d) 
   23533.00    6639.00       0.00     144.00    1976.61

I'm really not sure about what these lines above are doing, I appreciate any help on this!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
7
$\begingroup$

This is not what you do to validate an event time model. You need a smooth calibration curve at each of a series of time horizons plus validation of predictive discrimination, e.g., Somers' Dxy rank correlation (c-index). The R rms package makes this easy, and it can use the bootstrap to correct for overfitting if you are honest about including all candidate variables in the model. See my course notes for details: http://biostat.mc.vanderbilt.edu/rms

Improve this answer
$\endgroup$
5
  • $\begingroup$ So I cannot access how well the model predicts/ranks deaths before some time T? Can you explain to me what AUC.cd does? $\endgroup$
    – Fernando
    Commented Sep 9, 2016 at 14:45
  • $\begingroup$ @Frank Harrell, the OP is not trying to validate an event time model, he is trying to best use the data (as suggested in page 9 of your bookregression modelling strategies) .. ie he is interested in survival up to certain timeframes ( and probably wants to compare/benchmark against a logistic regression approach) $\endgroup$
    – seanv507
    Commented Sep 9, 2016 at 14:50
  • $\begingroup$ @seanv507 Yes, I'm very new to Survival analysis as explained, I'm even considering doing a Logistic Regression for each period, but I also would like to learn more about Survival models. $\endgroup$
    – Fernando
    Commented Sep 9, 2016 at 14:57
  • 3
    $\begingroup$ One who is interested in survival probability for a certain time frame $[0,t]$ is interested in whether the model is well calibrated for predicting the probability that the event occurs later than time $t$. And it is not appropriate to use logistic regression for time-to-event data. That would not be efficient nor would it handle variable follow-up time. $\endgroup$
    – Frank Harrell
    Commented Sep 9, 2016 at 21:15
  • $\begingroup$ Thanks. Indeed, logistic regression doesn't seem appropriate. Meanwhile I'll try to understand 'calibrate.cph'. $\endgroup$
    – Fernando
    Commented Sep 9, 2016 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.