I have a retrospective EHR database from a hospital and I would like to build an ML model to predict whether a patient will die within 28 days or not (from discharge/some time point T)

Can I check with you on the below steps?

a) Let's say I have a sample of 5000 patients

b) I choose training data as 3500 patients

c) For these 3500 patients, I extract the required variables and their values till time "T"

d) And now for training data, we need labels, So I calculate using a python script whether this person has died within 28 days from time "t" (because I have all their data. I can find the difference between his discharge date and death date)

e) If yes, I will label it as "1" else "0"

f) I build a supervised learning model using logistic regression

Now comes the challenging part (for me atleast)

g) I would like to apply this model on the unseen set of 1500 patients

h) I extract the same variables as training data

i) Now I apply the model to this unseen data of 1500 patients

k) But the problem is this will only give whether the patient will die or not. How can I know whether he will die or not in the 28 days?

How can I incorporate this time component here.

Can somebody help me with this by providing easy to understand steps and which algorithm to use please?

  • $\begingroup$ Why not do survival analysis? $\endgroup$ – DWin Nov 25 '20 at 0:49
  • $\begingroup$ @DWin - Can survival analysis help in prediction as well? Sorry, not done much with survival analysis before...will read up online... Any other ML algorithms that can help? $\endgroup$ – The Great Nov 25 '20 at 1:24
  • 1
    $\begingroup$ Indeed survival analysis can predict. There are many flavors of survival analysis. The survival package in R is a "recommended package, but there are also alternatives. I use both survival and rms. There's a well supported eha (event history analysis) and more recently the flexsurv package has become available. There is also a random survival forest package if you wnat to stay on the "data science" side of mythology (ooops, that was what my spell-chucker offered and it amused me) and don't want to stray into "real statistics". $\endgroup$ – DWin Nov 25 '20 at 4:04
  • $\begingroup$ LSTMs can work for such problems? $\endgroup$ – The Great Nov 25 '20 at 10:32

As @DWin says in a comment, this is a standard application of survival analysis. That has the advantage of evaluating not only event occurrence but also the times to events.

If you do proceed with your train/test split (even 5000 patients might not be enough for that to be reliable; model evaluation by resampling might be better) you say that you have time-to-event data. In that case, even with an all-or-none logistic regression model, you can evaluate 28-day mortality by ignoring deaths that occur after 28 days from your start time.

The rms package in R, also noted by @DWin, provides a well developed infrastructure for combining survival analysis with resampling-based model validation and calibration, and providing predictions for specified combinations of covariate values. There's no reason why the approaches used there can't be extended to analysis with more of a "machine-learning" or "data-science" flavor. Just be aware that models provided by those latter approaches can be harder for mere human beings to interpret.

  • $\begingroup$ Hi @EdM - One quick question. I was reading this paper translational-medicine.biomedcentral.com/articles/10.1186/… which tackles a similar objective as mine. Here I see they haven't used Survival analysis. But can time to event analysis be done using regular classification algorithms? Its just that they have derived the labels as "1" and "0" based on 30 day interval. Can help me please with this? $\endgroup$ – The Great Dec 17 '20 at 4:31
  • $\begingroup$ @TheGreat it's certainly OK to focus only on things like 28-day or 30-day mortality if you don't care about the actual times to events. If you do care about times to events, it's possible to use methods like those in your cited paper to evaluate survival models, too. For example, the R xgboost package allows for Cox and accelerated failure time survival models. Your choice comes down to whether or not you care about times to survival or only about yes/no survival up to a particular cutoff time. $\endgroup$ – EdM Dec 17 '20 at 15:50
  • $\begingroup$ Hi @EdM - Q1) Am a bit confused. Isn't focusing on things like 28-day or 30-day mortality called Time to Event analysis? Because based on your comment - it's certainly OK to focus only on things like 28-day or 30-day mortality if you don't care about the actual times to events, am I incorrect to understand that predicting 28 day or 30 day mortality is Time to event analysis? May I know why isn't that called as 'Time to event' analysis $\endgroup$ – The Great Dec 18 '20 at 16:00
  • $\begingroup$ Would you mind to help a noob like me to understand the difference/elaborate your comment - Your choice comes down to whether or not you care about times to survival or only about yes/no survival up to a particular cutoff time with an example please? I thought both that you mentioned are time to event analysis and there is no difference between them. But looks like there is some difference $\endgroup$ – The Great Dec 18 '20 at 16:02
  • 1
    $\begingroup$ @TheGreat yes, you have it right. If it's just survival up to some time, without reference to the actual time during that interval, then it's a classification (better, event-probability) task. If you want to model the actual time until an event, then it's a survival model. Besides survival:cox, the R implementation of xgboost (at least) also provides an accelerated-failure-time objective function (survival:aft) for survival models. $\endgroup$ – EdM Dec 20 '20 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.