Evaluation of survival analysis - time to event calculations and cutoffs

Question

I'm working with a survival model to evaluate its accuracy on a new dataset that was not involved in training or initial evaluation. This dataset is a truly independent dataset. Within this dataset, most of the events occur within just under 2 years (>90%), but we have additional observations that occur beyond that 2 year timepoint. Additionally, the median time of follow up is just under 2 years.

At this stage, I have chosen to calculate a 2-year risk score from the model for this new dataset, but there have been questions regarding how to properly identify those observations for which an event happened soon after the 2 year mark. Below are the two main approaches that I've tried to present in an unbiased manner.

One argument is that events are events, and given their close proximity to the 2-year mark, they should be considered events for all further evaluations as their 2-year risk scores are likely higher than any non-events observations within the study.

The alternative argument is that those observations for which events occurred after the risk score time should be counted as non-events regardless of their proximity to the calculation date. In other words, if an event occurred at 2 years and one day, that's still a non-event at 2 years and should be considered as such.

A third alternative consists of having three groups: non-events, events within 2-years, and events after 2-years.

Another suggestion has been to calculate the risk score for the maximum follow up time for an event observation, in this case 2.4 years thus covering all possible event times. The argument against this is that some of the censored (and thus non-event observations) may have had events that we do not know about, potentially altering the non-event group's risk scores.

I would appreciate any suggestions regarding how to approach this problem.

Some extra details following the response by Todd:

The original model is an AFT model and the original data had a 5-year follow up time. I'm using this model now in a new dataset which has a shorter follow up time overall, but is similar in composition to the original dataset. This is where the 2-year value for risk prediction is coming from, and it is simply a parameter within the AFT model.

For validation we have taken various approaches including test of calibration including: Hossmer-Lemeshow test using quantiles of risk scores compared to actual event rate within those quantiles; and most relevant to this questions, a discrimination test (discrimination slope?) comparing the distribution of risk scores for those observations considered non-events to those considered events.

Answer 1

I will address your question regarding inclusion/exclusion of events near the 2-year follow-up time and using all follow-up time. However, your description of data with very close survival time at the median and 90th percentile and the approach to validation are both intriguing as well.

I would take the view that survival analysis is designed to assess whether the probability of an event time (T) is greater than some observed time (t):

S(t)=Pr(T>t).

Counting events occurring after 2 years as occurring at 2 years is not a valid approach, especially when comparing to a derivation model that likely did not treat events in this way. If "events are events" then we could use a logistic analysis and glaze over the time-to-event. Assuming your are interested in the time to event, event times should not be altered based on their proximity to a selected end-of-followup date or duration.

A key assumption of survival analysis is that all persons/units censored have the same event rate as those persons/units who continue to be observed in the model (noninformative censoring). If persons/units who are more likely to fail are more likely to be censored, then bias is introduced that falsely increases measurement of survival. Perhaps a solution to all your concerns, and one that you allude to, is including all available follow-up time. As long as your censoring is non-informative, this should not bias the non-event group in any way.

While using all follow-up time may not bias derivation of a novel survival/hazard model, this strategy may/may not be applicable to a validation analysis. More information on your strategy would be helpful in determining the approach given your specific data and hypothesis.