How does time specific hazard ratios relate to time dependent incident dynamic ROC AUC?

Question

I have a question related to time specific hazard and time dependent incident dynamic ROC analysis.

I am currently working on some analysis where I evaluate a prognostic model (Cox model) on some survival data.
The plots I show here are based on the models prognostic index. Firstly the model yields significant p-value in the Schoenfeld test of residuals. So my interpretation was that it does not satisfy the assupmtion of proportional hazard. I also plotted the log Hazard for different time periods, using Harrels RMS-package, and to my surprise, the log Hazard did vary but it seemed to increase with time, which is quite counter-intuitive as I would expect the Hazard to decrease over time for a marker or a linear prognostic model. I also performed a incident dynamic time dependent ROC analysis using the RisksetROC-package. Here the AUC decreased over time when using the Cox estimator-method. It was more flat when performed with a Schoenfeld estimator-method, but the i-AUC-metric (integral under the curve for the whole time period) was lower when compared to the Cox-estimator-method.

My understanding is that the time specific hazard ratio reflects the relation between the individuals in the sample at risk based on the prognostic index in a manner that is similar to how the I/D ROC-analysis discriminates between cases and controls in the methods used in the RisksetROC-package. (" sensitivity measures the expected fraction of subjects with a marker greater than c among the subpopulation of individuals who die at time t, while specificity measures the fraction of subjects with a marker less than or equal to c among those who survive beyond time t." According to Heagerty and Zheng in "Survival Model Predictive Accuracy and ROC Curves" from 2005 where they describe the methods which "RisksetROC" are based on.

These findings are somewhat confusing to me:

1. I was surprised to see that the log-Hazard increased over time.
2. The time specific I/D-ROC AUC decreased over time (With the Cox-estimator), which I would expect to more or less follow the trends in the hazard ratio. What am i missing? Are these findings to be expected?

There are limited number of events at later stages, could this be an explanation?

Answer 1

There are a few somewhat related but separate issues here.

First is the apparent violation of proportional hazards (PH). Although the test showed a "statistically significant" violation of PH, the plot of variance-scaled Schoenfeld residuals over time still looks pretty flat at a log-hazard value a bit under 1. Consider whether that violation is large enough to matter in practice for your application.

Second is your interpretation of the time-specific hazard ratio (HR) plots. Each point is calculated based only on events that occurred within the associated time window. Although there seems to be an upward trend with time (consistent with the apparent PH violation), note the wide confidence intervals (CI) at the later time points due to the small numbers of events evaluated. The lower CI estimates at all times seems to be below even the lowest point estimate of HR at the earliest time, so I'd be reluctant to put too much importance on the trend. And even if the trend is real, I've been fooled too often by Mother Nature to be "surprised" at things like how hazard ratios evolve over time.

Third is a potential confusion between hazard ratios (HR) and discrimination as measured by AUC or concordance. A high HR indicates that some sort of average survival is shorter with a high level of that predictor, but it doesn't tell you how well that predictor discriminates between individual case pairs. A concordance/AUC index evaluates that discrimination. A high HR doesn't necessarily mean high discrimination, or vice-versa.

Fourth, there is no need for trends in the time-dependent ROC AUC values to match the trends in estimated HR. That's not only because of the argument in the previous paragraph. The Heagerty-Zheng approach weights each individual event-time ROC-AUC by the corresponding survival probability and the local density of events near the event time. That tends to down-weight later event times. The time-specific HR calculations do no such weighting.

Fifth, the drop in the Cox-model estimate of ROC-AUC could in principle be due to the PH violation, among other things. As I understand the Schoenfeld estimator used in that package, it models a time-varying coefficient for the predictor to get a better estimate of the ROC-AUC over time. The Cox-model estimate assumes a time-independent coefficient for the predictor. If you think that the PH violation is important, then you should be using the Schoenfeld estimator, which corrects for the lack of PH, rather than the Cox.

In your case, the Schoenfeld estimator is pretty consistently at or above an AUC of 0.7 at all times. The Cox-model estimator gives high discrimination at early times but drops off below an AUC of 0.7 after about a time of 10. I suspect that the higher "i-AUC-metric" for the Cox model has to do with the restricted integration out to the last event time. That gives it an advantage due to its apparently very high discrimination at early times. If you had events at later times, it's quite possible that the"i-AUC-metric" for the Schoenfeld estimator would surpass it.