At one point in David Spiegelhalter's The Art of Statistics, he describes Predict 2.1, a piece of software that helps decide, for a woman getting breast cancer surgery, which additional post-surgical treatment she should receive (if any). The software takes details about the woman (e.g. age) and her tumor (e.g. size), applies a certain algorithm, and provides estimated five– and ten-year survival rates with each of the different options.

This algorithm was based on data from 5,700 historical cases in the UK Cancer Registry. However:

But care is required in analysing the outcomes of women given these treatments in the past: they were given the treatments for unknown reasons and we cannot use the apparent benefits observed in the database. Instead a regression model is fitted, with survival as the outcome, but forcing the effect of treatments to be those estimated from reviews of large-scale clinical trials. [first US edition, chapter 6, page 182]

So, I don't understand; it sounds like we're ignoring the outcomes from those 5,700 cases in favor of those from clinical trials, but in that case, why even mention those 5,700 cases? How do they affect the algorithm if we're ignoring the outcomes?

I tried searching online for more details on how Predict 2.1 was developed, and I did find some (especially its History page), but not these sorts of details.

(I'm not particularly expecting anyone here to be specifically familiar with the details of Predict 2.1, but I'm hoping that the answer will be obvious to anyone with a better understanding of statistics. I imagine I'm misunderstanding something basic.)


As the Predict website says:

the cumulative hazard is the product of three components: the baseline hazard (chances of dying from something other than breast cancer), the hazard ratios for the risk factors (the increased risk of death due to breast cancer) and the hazard ratios for the treatments (the decreased risk thanks to the treatments).

The data from the UK Cancer Registry were used to determine the first two components: the baseline hazard for dying from something other than breast cancer and the hazard ratios for the risk factors. Those risk factors were estimated separately for patients whose tumors were positive or negative for estrogen receptor expression (ER+ and ER-). In both ER classes, age at surgery, the tumor size, the number of lymph nodes with cancer, the tumor Grade, and expression of the HER2 gene (a receptor for epidermal growth factor) are used as predictors. For ER+ tumors, whether the cancer was detected through routine screening and the expression of Ki67 (a measure of cancer cell proliferation) are also considered. The original source of data for determining the associations of those predictors with outcome was the UK Cancer Registry, although it seems that additional sources have been used since then.

As your quote from Spiegelhalter notes, data like those from the UK Cancer Registry can be unreliable for determining associations of specific therapies with outcome. Not only are registry data often incomplete in their annotations of therapy, but also the choice of therapy is often determined in part by the severity of disease, with less-aggressive therapies generally used to treat patients with early-stage disease. That makes it hard to disentangle the effect of a particular therapy from the other risk factors associated with the choice to use it.*

The most reliable way to determine the association of a therapy with outcome is a controlled randomized trial. Then all the other factors that might be associated with outcome tend to be averaged out between the treatment groups, directly exposing how the treatments themselves are associated with outcome. Such randomized trials, however, include many fewer cases than a cancer registry.

So: is it possible to combine information from the large-scale cancer registry data on risk factors with the rigorous assessment of associations of therapy with outcome available from smaller controlled randomized trials? The answer comes from the way that hazards combine in a Cox proportional hazards model.

In a Cox model, the associations of the predictors with outcome are additive on the log-hazard scale (multiplicative on the hazard scale). So it's reasonable to try to combine the terms for baseline hazards and cancer-associated hazards from the larger UK Cancer Registry with separate terms for effects of therapy that have been determined from randomized trials. Predict now includes estimates of effectiveness of 6 different types of therapy.

Combining predictor terms from those two different sources of data does depend on some assumptions. Nevertheless, the algorithm has been validated in independent data sets both in terms of calibration (how close estimated survival probabilities are to observed survival) and discrimination (whether the predicted survival order of 2 individuals agrees with the observed order).

*For an extensive discussion of the difficulties in trying to determine true treatment effects from observational studies, see the Causal Inference Book by Hernán and Robins.

  • $\begingroup$ Thanks; that makes a lot of sense! But, one part that I don't get: you say that "registry data [are] often incomplete in their annotations of therapy". But doesn't this sort of regression depend on knowing what therapy a given patient got? (How can you interpret that patient's survival data if you can't adjust for the effects of whatever treatment she received?) $\endgroup$
    – ruakh
    May 11 '21 at 3:36
  • $\begingroup$ @ruakh in practice the information in a registry has much of what's needed for prognosis. Tumor size, spread to nodes, markers of tumor aggressiveness, and age are major factors. Outside trials, standard-of-care therapy is chosen based on such factors. If one therapy is clearly better in a certain situaiton, then that will tend to be used in practice. If there are competing therapy choices, they are likely to have outcomes so similar that you need a randomized trial to distinguish. In that case, incorporating therapy into the model won't generally make much of a difference. $\endgroup$
    – EdM
    May 11 '21 at 14:43
  • $\begingroup$ So, I've thought about this off and on for a few weeks now, and I still don't entirely get it, but I think I get it as much as I'm going to for now. (I think I need to spend more time learning about Cox regression before this will fully make sense.) Thanks again! $\endgroup$
    – ruakh
    May 31 '21 at 8:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.