0
$\begingroup$

I have created a cox proportional hazards regression model for predicting churning out in R(using coxph function from the survival package).

I am able to see the churning out percentage with number of days of all my customers by plotting the survfit model of the above coxph model.

fit_cox <- coxph(formula = form, data = train)
fit_cox_model <- survfit(fit_cox)
plot(fit_cox_model,ylim=c(.4,1), xlab = 'Days since became member', ylab = 'Percent Surviving')

Now I had to predict the probability of each customer churning out after 30 days, 100 days and 1000 days. So I used the function predictSurvProb from the pec package.

prob_df <- data.frame(predictSurvProb(fit_cox,newdata=test_set,times=c(1,30,100, 1000)))

With that I am getting the probabilities, but I am not sure how do I validate this model. I just tried to manually compare the churned_out binary variable with these probabilities which obviously is wrong (But there are so any customers who have already churned out in 60 days but still there probability of surviving in 100 days is high like 0.9).

I even tried validate from rms package, and cindex from pec package for validation but they both are giving the following errors-

> validate(fit_cox_model)
Error in UseMethod("validate") : 
  no applicable method for 'validate' applied to an object of class "c('survfit.cox', 'survfit')"
> cindex(list("cox1"=fit_cox), formula = form, data = test_set)
Error in histformula[[2]][[1]] : 
  object of type 'symbol' is not subsettable

So I want to know -

1) Whether I am following the right procedure for predicting probability of churning out?

2) How do I validate my model ?

$\endgroup$
0
$\begingroup$

Regarding

2) How do I validate my model ?

You can use predict with type = "expected" to get an estimate of the expected survival time. Next, you can use survConcordance to get a concordance estimate if you put the expected survival on the right hand side of the ~ in the formula. You can cross validation concordance to get an idea of the generalization concordance.

The above will only tell you how good your model is at ranking customer. A second question is the absolute probability. For this, I see no reason why it should be "obviously is wrong" to perform cross validation of absolute churn probability at fixed points in time regardless of the baseline chance of churn at any given point. I am not aware of (/cannot recall) methods to do this over an entire survival curve though.

$\endgroup$
  • $\begingroup$ You can also turn to likelihood based methods (e.g., AIC and BIC), of course. $\endgroup$ – Benjamin Christoffersen Oct 16 '17 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.