General considerations
The most sensitive ways to compare Cox (or other) regression models overall are based on (partial) likelihood methods. It's also important to evaluate the calibration of the model, in terms of how well predicted and observed outcomes agree (at some particular point in time for a survival model). See this post by Frank Harrell.
Receiver operating characteristic (ROC) or precision-recall (PR) curves aren't so sensitive as those preferred methods for model comparisons. They are both rank-based displays that don't take calibration into account.
To a great extent the limitations of ROC curves carry over to PR curves, as if you have an ROC curve you can generate the corresponding PR curve from the corresponding numbers of true positives (TP
), true negatives (TN
) and false positives (FP
) at each point (predictor value) along the ROC curve. "Recall" for a PR curve is just a synonym for the "true positive rate" or "sensitivity" plotted on the vertical axis of the ROC curve. Precision is TP/(TP+FP)
. (True negatives, TN
, aren't used in PR curves, so you can't go from PR to ROC.)
The area under the ROC curve (AUROC) in binary-outcome models, often used nevertheless to compare models, is equivalent to the probability that the model puts 2 randomly chosen cases into the correct order. Harrell's c-index for survival outcomes is effectively equivalent, as it's the fraction of comparable cases (taking censored event times into account) that are put in the correct order. In the post linked above, even Harrell doesn't recommend using the c-index for comparing models, just for documenting a particular model's ability to discriminate cases.
The area under a PR curve (AUPR) is considered an "average precision," but that interpretation needs to be tempered a bit by the curve's nature. There's a whole region of PR space that is unachievable for a given data set, determined by the event probability. See for example Boyd et al., Proceedings of the 29th International Conference on Machine Learning, Edinburgh, Scotland, UK, 2012. That sets a lower limit to the "average precision" that needs to be taken into account.
When there's a low event probability, differences between models are admittedly more evident in AUPR than in AUROC; see Saito and Rehmsmeier, PLoSone 10:e0118432, 2015 and Zhao et al., Diagnostic and Prognostic Research 5:13, 2021. I'm not aware, however, of any evidence that the AUPR is superior to proper model calibration in this low-probability regime.
If you really want to compare PR curves, despite the above, I'd be reluctant to depend on any theoretical, model-based statistic. Bootstrapping would seem to be the best choice. Even that might be problematic, as the lower limit to AUPR might prevent AUPR from being a pivotal quantity suitable for bootstrap estimation. I suspect that "BCa" (bias and acceleration corrected) bootstrapping would be needed.
Time-dependent curves
As noted above, once you have an ROC curve you can generate the corresponding PR curve. That's also true, in principle, for time-dependent ROC curves. Specifics with censored event times will depend on the type of smoothing or weighting used to generate the curve and on which type of time-dependent curve is generated. Kamarudin et al. review those types of time-dependent curves and smoothing/weighting methods in BMC Medical Research Methodology (2017) 17:53. See Figure 1 for a simple outline of how the numbers of cases evaluated depend on the predictor value along the ROC curve and the type of curve; you'll need that to parcel out the TP
and FP
numbers if those aren't immediately provided by the software you use.
Note that the timeROC
package that you cite has a SeSpPPVNPV()
function that can be used to estimate time-dependent "sensitivity" (aka "recall") and "positive predictive value" (PPV, aka "precision") that you would need to generate PR curves directly from your data. Again, if you really want to compare AUPR values between models, bootstrapping (with the above cautions) would be my choice.