1
$\begingroup$

I would like to use concordance to have an idea of my models discrimination power. Because I have time dependent covariates, the time to event data is in a "long" format, i.e. one row per time unit per individual. (dataset approx 1.5million rows)

I use the R package survival, which provides a concordance measure in the model output, as well as a concordance function. I obtain unexpectedly high concordance estimates and very narrow confidence intervals (e.g. C=0.81 se=0.001)

1- I am wondering if the estimates are correct with data in the long format. At the very least I reckon the standard error is artificially lower because of the number of rows?

2- If I wanted to calculate concordance manually going back to the subject level, I have the time-to-event, but what risk estimate can I use? (as it is time dependent)

3- There is also a frailty term in my model. Are concordance calculations still valid in this case?

I've looked in the package documentation but can't find any mention of these issues.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Much of this is explained in detail in the concordance vignette that accompanies the survival package. The concordance() function does allow for what you call "long form" data (Surv objects of the counting type, with stop and start times and a binary event indicator, and potentially many rows per case). The way this is handled is described at the end of Section 3 of the vignette:

at each event time the current risk score of the subject who failed is compared to the current scores of all those still at risk

which shows how you could proceed with your own "manual" calculations. You can also examine the source code for the concordance function (although the heavy computational work is in C code). Certainly, with a very large number of events (event times are used as the starting points for concordance calculation) you would expect to have a very small standard error in the estimate of the concordance.

The concordance calculation is based on the specified start and stop times, event indicators, and linear-predictor values from the model. I don't know for certain (you could check the code), but I suspect that the concordance calculation thus proceeds similarly to the way that the predict.coxph() function handles frailty terms when evaluating new data: "the predictions will always be for a random effect of zero."

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1. In general, one should proceed with extreme caution when looking at predictiveness but allowing for time varying covariates. The point is that a prediction (in the context of concordance and discrimination) is fundamentally cross-sectional. Otherwise, you wind up on paradoxes; such as smoking doesn't predict cancer risk, but presence of abnormal lesions and nodes on CT is highly predictive of cancer. $\endgroup$
    – AdamO
    Nov 15, 2022 at 18:03
  • $\begingroup$ Also how bizarre the "concordance" example returns the AUC for a binomial GLM - is this a conflict of definition about what concordance is? We often discuss the Hosmer Lemeshow test as a measure of concordance in the binomial GLM. $\endgroup$
    – AdamO
    Nov 15, 2022 at 18:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.