5
$\begingroup$

Tissue microarrays are commonly used to assess potential prognostic biomarkers. For decades now, many authors (I would even say the majority) feel the need to categorise their continuous predictors, either by dichotomising after data collection, or by using scoring systems that pre-categorise (e.g. Allred and immunoreactive score).

The pitfalls of this approach are numerous and have been highlighted for years. Douglas Altman and Patrick Royston have been some of the most vocal critics, alongside prediction model experts like Frank Harrell.

I completely agree with their arguments, but does it really matter in practice? A frequent defence of dichotomisation, or categorisation of continuous predictors in general, is that it greatly simplifies clinical decision making. Indeed, a physician may have to weigh up what is essentially a binary choice – to treat or not to treat; or to present the patient with one prognosis or another – and so the belief is that predictors should be dichotomised to reflect this. I would argue that keeping a predictor continuous, however, allows for more granular information to be obtained and lessens the risk of misclassifying patients whose predictor values lie just above or below the arbitrary threshold.

However, countless clinical decisions are made off back of categorised predictors. For example, breast cancer patients are assessed for oestrogen, progesterone and HER2 expression status and are classified as receptor positive or negative as per some threshold. Although this approach may not be 'statistically sound', its widespread use in guiding decisions over treatment surely points to categorisation at least being somewhat useful?

$\endgroup$
2
  • $\begingroup$ Yes, it is that bad. As I argue here dichotomizing can yield biased predictions, meaning the dichotmomization is good for some but can be bad in many more cases. Risk estimates are either too high or too large, which effects decision quality. $\endgroup$ Jan 17 at 17:59
  • $\begingroup$ @DemetriPananos sometimes in medicine you do have true biological dichotomies, as in breast cancer. That becomes a classification rather than a continuous-outcome prediction problem, if the classes respond differently to therapy. The difficulty is then how to assign a particular case to the best therapy. In practice, pathologists might use a trichotomy for a marker: upper and lower cutoffs that have very high probability of correct assignment, with intermediate cases assigned to more testing or more cautious application of findings. $\endgroup$
    – EdM
    Jan 17 at 18:19

2 Answers 2

11
$\begingroup$

It's really that bad, as detailed in many of Patrick and my writings. Think about it these ways, for starters: If you categorize a marker the loss of information is so great that you will need to go an collect more markers to make up for the loss. Why not instead get the most information out of a single marker? Then there is the issue that it is easy to show that the threshold for a marker is a function of the continuous values of all the other predictors. See the Information Loss chapter in BBR. For example, if the sex of the patient is an important factor with regard to the likelihood of disease, you'll find that a different biomarker threshold is needed for females vs. males.

Categorization simplifies decision making but only by making it worse. Many things only seem to be simple precisely because they are wrong.

We are taught to assess goodness of fit of models. A model that assumes a piecewise flat relationship between a biomarker and outcome will be easily shown to have poor fit to the data. For example look at the log-likelihood for a piecewise flat model (i.e., one using thresholding) and a flexible but everywhere-smooth model.

$\endgroup$
4
  • $\begingroup$ @EdM and others. Thank you all for your contributions (sorry I can't tag you all in these replies). To give some context, I'm a PhD student looking at the use of Cox modelling in lung adenocarcinoma. I'm investigating the use of common prognostic variables (e.g. node metastasis, T-stage, tumour pathology) along with TMA biomarker data in the prediction of survival. $\endgroup$
    – user333336
    Jan 17 at 10:17
  • $\begingroup$ With respect to lung cancer, the field seems awash with models that pay scant attention to the correct handling of biomarker data. Indeed, I initially only read papers in this area and came away with the impression that categorisation was the way to go. It was purely by chance that I came across the writings of Frank Harrell et al. and thought that there might be a different way to handle my data. $\endgroup$
    – user333336
    Jan 17 at 10:17
  • 2
    $\begingroup$ The example I gave of breast cancer patients being treated according to pre-defined receptor expression thresholds was nicely expanded on by @EdM. Thinking about it a little further, I suppose that ER, PR and HER2 are more examples of predictive rather than prognostic biomarkers. Although the two are not mutually exclusive, the clinic utility of the aforementioned lies in their ability to discriminate different treatment groups. For the most part, and certainly where lung cancer prognostic models are concerned, categorisation of biomarker data is not the way to go... $\endgroup$
    – user333336
    Jan 17 at 10:17
  • 1
    $\begingroup$ @user333336 prognostic biomarkers can become predictive biomarkers, as for HER2 in breast cancer. Patients with HER2-overexpressing tumors were first found to have worse outcomes, leading to attempts to target HER2. That led to trastuzumab as (part of) therapy, and clinical trials that showed the predictive benefit of HER2 expression levels for that therapy. But with HER2 there is an order of magnitude difference between the small percentage of over-expressors and others; see the density plots I added to my answer. I don't think you'll find anything so dramatic in TMA of lung adenocarcinoma. $\endgroup$
    – EdM
    Jan 17 at 18:03
6
$\begingroup$

Although cutoffs in biomarker values are admittedly not helpful for modeling within a single population (+1 to Frank Harrell's answer on that basis), they make sense when, together with extensive subject-matter knowledge, they can delineate fundamentally different populations that benefit from different therapy. In such situations, testing can classify most cases with high probability (based on separate high and low cutoffs) while assigning intermediate cases to further testing or more cautious application of the findings. Breast cancer, the example of the OP in which tumors are evaluated for estrogen receptor alpha (ER), progesterone receptor (PR) and human epidermal growth factor receptor 2 (HER2), is such a situation.

HER2 vs ER in TCGA BRCA

This hexbin plot illustrates the 2-dimensional distribution of the expression of HER2 versus ER among 1093 primary breast cancers, based on RNA sequencing data from The Cancer Genome Atlas. Note the logarithmic scales on both axes.

Density plots for ER and HER2 make the class distinctions clearer.

density plots ER HER2

ER shows a clearly bimodal distribution with modal values differing by 2 orders of magnitude and few tumors having intermediate values. HER2 values show a major peak encompassing about 90% of tumors, but about 10% show substantially larger expression, up to 20-fold more than that of the major peak.

Decades of clinical and laboratory research have shown that these distributions represent fundamentally different biological classes of breast cancer, driven by different tumorigenic processes and responding best to different types of therapy. For example, with respect to tumors having high expression of HER2 (also called ERBB2), Wikipedia summarizes:

Amplification, also known as the over-expression of the ERBB2 gene, occurs in approximately 15-30% of breast cancers. It is strongly associated with increased disease recurrence and a poor prognosis; however, drug agents targeting HER2 in breast cancer have significantly positively altered the otherwise poor-prognosis natural history of HER2-positive breast cancer... HER2 is the target of the monoclonal antibody trastuzumab (marketed as Herceptin). Trastuzumab is effective only in cancers where HER2 is over-expressed.

Similarly, patients with high ER expression can benefit from "endocrine therapy" that interrupts cellular signaling pathways mediated by estrogens. But cases at the lower left of the hexbin plot represent "triple-negative" breast cancers, low in all of ER, PR and HER2, that do not respond to therapies that target either estrogen signaling or HER2 signaling.* They need different therapies.

In practice, expression of ER, PR and HER2 is evaluated at the protein level by immunohistochemistry (IHC) rather than by the RNA sequencing that provided data for this plot. IHC makes at best semi-quantitative assessments of protein expression. For example, the Allred score cited by the OP combines the intensity and extent of IHC staining into an 8-point scale.

Nevertheless, given the large differences between high- and low-expressing tumors with respect to levels of these particular proteins, it's generally possible to assign a tumor to one of those biologically distinct classes based on IHC. The current recommendation of the College of American Pathologists (CAP) is to use the percentage of tumor-cell nuclei showing ER expression as the criterion, with tumors having <1% called ER-negative and those with >10% ER-positive. Tumors having 1% to 10% on that scale are reported as "ER Low Positive" with specific cautions about therapeutic choices that might be based on that finding. Similar guidelines for HER2 suggest an additional type of assay, in-situ hybridization, when IHC results are equivocal.

In that respect, behavior in practice follows what Harrell says on page 258 of Regression Modeling Strategies: "Physicians sometimes profess to desiring a binary decision model, but if given a probability they will rightfully apply different thresholds for treating different patients or for ordering other diagnostic tests." Based on clinical outcome studies, the CAP guidelines identify levels of expression that have high probability of correct classification, and recommend further tests and caution otherwise.

Finally, the classifications based on ER, PR, and HER2 expression are only part of what's considered in choosing therapy. Those are considered together with the tumor's histological appearance and size, whether it has spread to lymph nodes or beyond, further gene-expression or genetic tests, and patient preferences for breast conservation to select a combination of surgery, radiation, and pre-operative and adjuvant (post-surgical) drug therapies. The National Comprehensive Cancer Network publishes a 200+ page book of Guidelines for breast cancer therapy based on such considerations, documenting the level of evidence behind its recommendations, with 700+ references to the literature.

Data and code:

From Firebrowse, select BRCA for breast cancer, click on the mRNASeq bar, and select illuminahiseq_rnaseqv2-RSEM_genes_normalized. Restrict to primary tumors (sample type 01). Put into a data frame, here called brcaRNAseqPrimDF. Then in R:

library(ggplot2)
## for hexbin
ggplot(brcaRNAseqPrimDF, aes(x=ESR1.2099, y=ERBB2.2064)) + scale_x_log10() + scale_y_log10() + geom_hex(bins=50) + xlab("ER") + ylab("HER2") + theme(aspect.ratio = 1)
## for density plots
pHER2 <- ggplot(brcaRNAseqPrimDF,aes(x=ERBB2.2064))+scale_x_log10()+geom_density()+xlab("HER2")+geom_rug(alpha=0.2)
pESR1 <- ggplot(brcaRNAseqPrimDF,aes(x=ESR1.2099))+scale_x_log10()+geom_density()+xlab("ER")+geom_rug(alpha=0.2)
library(gridExtra)
grid.arrange(pESR1,pHER2,ncol=2)

ESR1 is Estrogen Receptor alpha. ERBB2 is the currently accepted name for HER2.


*For simplicity I'm not trying to incorporate the third classic immunohistochemical marker, prostaglandin receptor, as essentially all cases with low ER also have low prostaglandin receptor expression (called PGR.5241 in the data).

$\endgroup$
7
  • $\begingroup$ Not a medical professional here, but this pattern of several clusters being over-simplified is UBIQUITOUS: seen it in plant biology, particle physics, material science... Doctors in particular won't really do do anything too complex, they have their own share of issues to be overwhelmed with. Better decision tree is a great boon but it HAS to be streamlined to be usable. Until there emerges an eponymously-named scale where you compare numbers coming from the lab with provided ranges and assign them numbers 1 to 5 and out goes the treatment plan and the prognosis, it goes nowhere... $\endgroup$
    – Lodinn
    Jan 15 at 5:31
  • 2
    $\begingroup$ @Lodinn I think you underestimate the ability and desire of clinicians to deal with complexity. I've edited the question to explain what is done when the classification scheme based on ER, PR and HER2 is equivocal, and how that classification is combined with other types of information to direct therapy choices. $\endgroup$
    – EdM
    Jan 15 at 17:23
  • 1
    $\begingroup$ @Lodinn the fundamentally respectful give-and-take among users on this site it one of its strengths. Thinking about your comment led me to improve my answer, and I learned more in the process. What I hoped to covey was a sense of the (limited) type of situation when biomarker cutoffs might be other than "bad." And you are correct about the variance in clinician competence. Admittedly, not all might follow NCCN and CAP guidelines or deal properly with classifications that aren't of very high probability. $\endgroup$
    – EdM
    Jan 16 at 15:03
  • 1
    $\begingroup$ The apparent clustering in the HER2-ER plane is actually a demonstration to the contrary point. There is no discontinuity in the data, and HER2 and ER have been arbitrarily dichotomized. This is why so much of cancer research involves finding new markers and subpopulations that are just making up for the discarding of quantification of HER2 and ER. If markers were captured using the fullest amount of information possible, I conjecture that subpopulations found by other markers tend to collapse. $\endgroup$ Jan 17 at 13:12
  • 1
    $\begingroup$ That all makes sense, and does not support torturing the data to come up with a cutpoint. $\endgroup$ Jan 17 at 17:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.