Lifelines-CoxTimeVaryingFitter for Multistate Survival Analysis

Question

I am new to survival analysis and cox regression, and have limited statistical background. I have time-to event data for a multistate survival model and I want to fit a cox model for each transition in the state model to figure out influence of different covariates on survival time. Some of my covariates are time-dependent, thus I am using CoxTimeVaryingFitter.

1-To fit the model per transition, I am only using the data from origin state to target state as well as the instances that were censored in the origin state. So, for both transitions 1->2 and 1->3, the censored data on state 1 will be used. Is this a correct approach?

2-I have around 2M instances for each transition, thus it takes a very long time to fit one transition. Can I do something like fitting the data in batches and taking the average of coefficients?

3-How should I use the penalizer parameter of CoxTimeVaryingFitter?

4-How can I test the validity of the model? For CoxPHFitter, I could look at concordance score and proportional hazard assumptions but CoxTimeVaryingFitter doesn't have that.

Answer 1

I'm not very familiar with the Python lifelines package, and questions that are very software specific are off topic on this site. Here's a bit of guidance, or pointers to guidance.

1. The vignette on "Multi-state models and competing risks" from the R survival package is a very helpful outline of how to deal with this type of situation. A preferred way to proceed is to model all transitions together in a single model, as that gives you the probability of being in any state over time. In some simple situations you get the same Cox regression coefficients if you model each transition individually and censor at times of transitions to other states, but I'm not sure whether that's true in general. Unless you have such a simple situation, I suspect that you would be better off using software that is designed for multi-state models.

2. It's not clear that there would be much net time saved by analyzing multiple separate batches. To get the final result you have to wait until all the batches are done. Would that be less time than waiting for one long complete run? Certainly, check that your code works OK on smaller batches before you run all the data. That said, the coefficient estimates in a Cox model are asymptotically distributed as multivariate normal. I suppose you could do multiple batches and average coefficients, but don't forget that you also need to have the corresponding coefficient variance-covariance matrix estimate in order to do inference on the coefficients.

3. The penalizer parameter seems to be related to a ridge-regression penalty. There's no need to use that unless you want to do that type of model.

4. The author of the lifelines package refuses to allow for predictions from models with time-varying covariates, for very good reasons outlined in this answer. Some types of model validation are possible with other software implementations, at least for types of validation that don't require predictions from the model. This answer outlines some possibilities. It sounds like your data set is large enough to allow for split-sample validation, if you use an implementation that allows for predictions from models with time-varying covariate values. But be very, very careful: in this type of situation you can easily end up making predictions about impossible scenarios or engaging in survivorship bias.