This is a question I originally posted on r-help but it is more suited here. I will post the question and the answer I received from Dr. Winsemius and would be most grateful for any additional answers you can provide.
I am evaluating survival models using Brier score (“peperr”) and Harrell’s C-Index (“Hmisc”). I am wondering:
What would be considered a “good fit” according to these scores (like the heuristic levels we have for R square in linear regressions) ?
Are there any papers to cite on the matter (I couldn’t find any) ?
Is there any paper to cite that discusses the limitation of using traditional reporting for model fit in survival analysis as opposed to these measures ?
Dr. David Winsemius replied:
Frank Harrell's excellent text "Regression Modeling Strategies" has an extensive discussion of "goodness of fit" and the principles of model comparison. It's both too involved as well as off-topic for Rhelp. The other text to consult is Steyerberg's "Clinical Prediction Models".
I predict that the RMS bibliography would be an excellent place to start your search.
Despite getting his name attached to what he calls the 'c-index', I don't think one could call Frank Harrell a proponent of that measure or any of the "competitors". It's really just a dressed up/transformed AUC. The message I have taken from reading his book and listening to presentations is that one should apply biologic tests of sensibility as well as careful investigation of the functional relationships between candidate predictors and the outcomes of interest. He speaks very disparagingly about automatic procedures.