First, don't worry too much about having a lot of "statistically significant" predictors if you have a large data set (as you presumably do with customer churn analysis). With a large enough data set almost any predictor might show a "statistically significant" association with outcome, even if it isn't of much practical significance. That's like the problem with normality testing: with a large enough real-world data set you will in practice tend to find "statistically significant" deviations from normality that don't fundamentally matter.
In terms of validating a model with time-varying covariate values, I don't know many details of what's possible in lifelines
, but in principle the tools for evaluating Cox models can be applied to models with time-varying covariates. The problem is that some of those tools implicitly involve making predictions. For example, an optimism bootstrap builds the model on multiple bootstrapped samples of the data and evaluates the models' predictions on the full data set. Predictions with time-varying covariates, however, aren't straightforward and can easily lead to circular reasoning and violations of causality.
The lifelines
author has put this quite clearly on another page on this site:
you can't predict for epistemological reasons... To predict, you must have time-varying $X$, that is, $𝑋(𝑡)$. But, if you have $𝑋(𝑡)$, then you must be making measurements on a customer, so they are not dead!
That said, some but not all survival tools in R can nevertheless make predictions from a time-varying covariate model. For example, the Predict()
function in the rms
package won't handle them, but its survest()
function and the standard survival
package survfit()
function can. The rms
validate()
function will perform an optimism bootstrap on such a model, although I couldn't readily get its calibrate()
function to work.
Standard tools for checking proportional hazards (PH) work with these packages (as I suspect they might in lifelines
, as the usual scaled Schoenfeld residuals test for PH just uses model-expected versus observed covariate values at event times and thus doesn't involve event predictions). I don't immediately see a reason why you couldn't use martingale residuals to evaluate the functional form of a continuous time-varying covariate (although I haven't done that myself and might be missing something important).
The question is just what is being predicted or validated. With data and analysis in the (startTime, stopTime, event)
counting-process format, these functions evidently treat each line of data as a separate observation, left truncated at startTime
with an event or censoring at stopTime
. That makes sense from the way that the data are modeled in situations with at most a single event per individual. As the R survival time-dependence vignette (a superb introduction to handling different types of time dependence in Cox models) says near the end of Section 2, in that case:
The likelihood equations at any time point use only one copy of any subject, the program picks out the correct row of data at each time.
So in that situation it makes sense to think about separate data lines as independent (albeit time-limited) observations.
It's possible (if you pay close attention to some data-formatting quirks) to get per-subject predictions from the predict.coxph()
function in the survival
package. Those seem to be based on predicting for all data rows and then summing all rows associated with a subject, however, so they might not make sense for all types of predictions. I'm not at all sure how well these methods will work with multi-state models involving time-varying covariates, in which intra-subject correlations need to be taken into account.
Finally, be very careful in applying these models to customer churn. A Cox model evaluates covariate values precisely at event times. A covariate associated with the specific time of loss of a customer thus might be like the following reverse-causality situation in clinical survival analysis (Section 3.5 of the vignette):
It is sometimes the case that a covariate measured soon before death is not a predictor of death but rather is simply a marker for an event that is already in progress... A simple example would [be] the time dependent covariate “have called the family for a final visit”.
If you are modeling customer churn in an attempt to prevent it in the future, you don't want to be modeling something like the "last straw" that finally pushed the customer to abandon your enterprise. With time-varying covariates, think carefully about what you are modeling, what you are validating, and what you are predicting.