When doing Cox regression, what should we take as event duration for censored data, and how should we divide the data into training and testing sets?

Question

I am starting out with survival analysis, and am confused about some things specifically in Cox regression. I have not found a tutorial that explains these things clearly, that's why I'm posting this question here, hoping for some clarity.

CONFUSION #1:

Say, I'm trying to predict time to death of some individuals. Specifically, I want to know the probability of death of each individual in the data set who is not yet dead after t days/months/years from the current day. The data set contains info about a number of people like age, ethnicity, income, date of birth, date of death and some other fields. If I want to use the Cox proportional hazards model for this, I would need to derive the time-to-event field, in this case TIME_TO_DEATH, right? How should this be done? For the people who have already died, I can calculate this as the difference between the DATE_OF_BIRTH and DATE_OF_DEATH columns; but how do I calculate this for the alive individuals whose DATE_OF_DEATH values are empty?

• Do I even need to calculate this - should it be left empty for the alive individuals?
• Do I assume the DATE_OF_DEATH for the alive individuals as the current day (or the last observation day for the dataset), in which case the TIME_TO_DEATH becomes identical to the AGE column?
• If these approaches are wrong (which I suspect they are), then what's the correct approach to calculate the TIME_TO_DEATH for the alive individuals? I have not found any article on Cox regression talk about this.

CONFUSION #2:

Assuming we have somehow resolved confusion #1, I would do something like this for the regression, right?

import pandas as pd
import lifelines

data = pd.read_csv('Dataset.csv') # contains age, ethnicity, income, date of birth, etc 
# the event duration column is 'TIME_TO_DEATH' - how to calculate this?
# the event column is 'DEATH' - 1 if death occurred, 0 if still alive 

cox = lifelines.CoxPHFitter()
cox.fit(df=data, duration_col='TIME_TO_DEATH', event_col='DEATH', show_progress=True)

Now, I would need to validate the model to know its accuracy, right? How do I do that?

• If I split the data into training and testing sets:

  • should I do a random split, so that the training and testing sets contain info about both dead and alive individuals, and then do cox.fit(df=data_train, ...)? In that case, how do I validate the model for the alive individuals (as I don't have the ground truth TIME_TO_DEATH values for them)?
  • should I take the dead individuals as the training set and predict for the alive ones? This way, I can at least validate the model (since I have the TIME_TO_DEATH values for the dead individuals), but if the number of dead individuals in the data set is, say, only 20-30%, then it'd probably not be a good model (as typically we take 70-80% of the data as training data for most machine learning models).

• Should I do something similar to A/B testing, where I construct the training data set by filtering the data to contain info up to t time, and the testing data set to contain info after t time until now? What I mean is, say t=3 months. Then, should I filter out the records in the data set that happened in the last 3 months and train the model on that, and then evaluate the model on the records that happened in the last 3 months?

• Another idea that I can think of, although it won't be Cox regression anymore, is to divide the time period into n divisions, and doing logistic regression on each of these divisions to answer with a yes/no to the question, "will this individual die in the i*nth time interval, where i={0,1,2,...}?" That is, if say n=6 months, will this individual die in the coming 0 months (i.e., immediately), 6 months, 12 months, 18 months, etc? The only problem with this method that I can think of so far, apart from having n models for each of the n time divisions instead of 1 model for all the data, is sampling noise - if there is a very less number of records in one or more of these time intervals, then the model for those time intervals wouldn't be too robust. Consequently, we wouldn't be able to make n too small. For example, we couldn't have n as 1 week in my data set, as there are only 5-10 people who die each week.

Answer 1

Answer to you CONFUSION #1:

For survival analysis, need to variables putting together to form the response, one is time-to-event, another is indicator for event of censoring.

For dead one, the difference between the DATE_OF_BIRTH and DATE_OF_DEATH is the value for time-to-event variable. The indicator variable = 1 to indicate that event (death) happened.

For people without death information, time-to-event the difference between the DATE_OF_BIRTH and the latest date that you know this person was alive. It cannot be the current day. For example, you know this person was alive on 30JUN2018, maybe this person died at 31OCT2018 and you do not know. So use the the difference between the DATE_OF_BIRTH and 30JUN2018 as time-to-event value. The indicator variable = 0 to indicate that event (death) did not happened.