Deciding Optimal Cutoff for a Prognostic Index derived from Cox Proportional Hazards

Question

I am planning to develop a prognostic model that would identify a particular group of head neck cancer patients who will do better if chemotherapy is added to standard radiation therapy. The data for the patients in question is derived from a randomized controlled trial. I have developed a prognostic model following the methods proposed by Dr. Harrell and Dr. Steyerberg and several posts in Cross Validated. The model was derived in patients who received radiotherapy alone and on application on patients receiving chemotherapy along with radiation it shows that the outcomes are different. I have used the median value of the prognostic index to divide my patient population into a group with good prognosis and a group with poor prognosis. I can see that the addition of chemotherapy makes a large difference in patients with poor prognosis but not in patients with good prognosis.

What I want to know however is how to determine the optimal cutoff for the prognostic index at which this difference becomes large enough that the physician will prefer to add chemotherapy (primarily as giving it in all patients is associated with the issues of excess toxicity with little additional benefit). One way I have thought of is deriving the Number needed to treat for each cutpoint - for each value of cutpoint calculate the benefit that addition of chemotherapy provides in the poor prognostic group and then calculate the NNT. Then use a predefined NNT threshold to determine the cutoff for the prognostic index. This, in turn, can be used to derive the optimal cutoff point on the nomogram that I will develop using the prognostic model.

My question is if this approach is a sound and if a better approach is available?

Answer 1

You should refrain from setting a pre-defined cutoff. Think instead of your work as providing information that a physician and patient can use to choose whether to add chemotherapy, based on the risks and benefits that apply in individual cases. Individuals may differ in the tradeoffs they will accept between potential prolongation of life with chemotherapy and the associated risks. Your data and analysis, with its prognostic index that presumably combines several prognostic variables, provide a well-defined place to start in assessing those tradeoffs.

For each value of your prognostic index you have some measure of the expected added benefit of chemotherapy and of your confidence in that expected benefit. That measure may be NNT if you really are talking about cures, or months of additional survival if these are patients with advanced disease. Concentrate on that continuous measure of the expected benefit of adding chemotherapy, and on the reliability of your estimates.

Two patients with the same prognostic index may have different risk-benefit tradeoffs, or clinical characteristics not included in your model might indicate whether a particular patient could tolerate chemotherapy. Such issues must be considered in addition to the predictions of your survival model.

You seem to be pursuing a high-quality approach to this difficult clinical problem. Do not undercut your work by jumping to a cutoff that will lose detailed information that is needed for intelligent decision-making in individual cases.