# Evaluating a survival model when focus in on "real time"

Assume a pure prediction problem.

Say I want to evaluate a prediction-focused survival model in the context of actual dates instead of at a specific survival time or integration thereof.

What would be the correct way to approach this/which metrics would be the correct measures of calibration/discrimination in this case?

An Example:

Assume a survival model based on only right-censored data. Time-since-entry is measured in months. I obtain $$\hat{S_i}(t_i|X_i)$$. Calculating this survival probability for each $$i$$ with respect to a specific date (e.g. january year X), makes it necessary to use different $$t_i$$ for each $$i$$ (for the population still at risk at this specific date). Of course, a little rounding bias gets introduced due to calculating survival times and then translating them back to real time.

1. How to calculate the (time-dependent) AUC and the Brier/Graf Score in this setting?

2. Specifically, does one even need weights of the censoring distribution for the brier score/AUC? (see e.g. here)

It makes the most sense to evaluate the model with respect its original time origin, with time = 0 as study entry date. If the model's performance differs depending on the calendar date of time = 0 for an individual, you probably haven't incorporated calendar date of entry appropriately into your model. Check that first, and fix if necessary instead.
• @RobG. define time = 0 for survival as the calendar entry date and the survival times as times relative to that date. Include the actual entry_date (e.g., months since January 2000) as a continuous predictor in the model. Model entry_date flexibly, for example with a regression spline. That will adjust for any calendar-date-related changes in survival behavior (as seen in clinical studies where therapy improves over time). Do your AUC/Brier evaluations on that model, in its own time scale.
• @RobG. when you do a prediction for survival of an individual through a particular calendar date, enter the corresponding entry_date value as a covariate along with the other covariates in your model, and make a prediction for the number of months between that entry_date and the calendar date of interest.