10
$\begingroup$

As part of a collaboration, I've been asked to fit a model with a continuous response $Y$ and an ordinal predictor $X$ (levels 1 to 5). The dataset owner is after an answer that is inherently binary: how does $Y$ differ between $X$ = 1/2/3 vs. $X$ = 4/5? Their reasoning for the binning is that:

  1. The instrument is measured poorly and is in reality more binary i.e. there is a meaningful, large change between 3 and 4.
  2. The model will be used to decide whether to take an action, and this is a yes/no question.

I'm on board with the notion that one should not bin ordinal or continuous data because it throws away information. But what's the best way to proceed here? Are there other ways to think about and approach this beside the two extremes of treat-as-binary vs. treat-as-ordinal? For example, is it possible to fit an ordinal model, and then combine the model coefficients in a way that addresses the binary question?

In case it matters: I work in R and there is also a covariate $X_2$ that interacts with the ordinal predictor.

Improve this question
$\endgroup$

3 Answers 3

Reset to default
12
$\begingroup$

Binning X will make problems 1 and 2 worse. Comparison of X=1-3 vs. X=4-5 is inherently ill-defined and uninterpretable unless the relationship between X and Y is flat when X=1-3 and when X=4-5. Make whatever specific comparisons are of interest, e.g., X=1 vs. X=4.

The real issue is how to model X. Treating it as an unordered categorical variable is inefficient but will definitely fit the data (X would take 4 d.f.). I often approximate this with a quadratic fit in X with 2 d.f. The only solution that really respects the ordinal nature of X is provided by the Bayesian modeling R package brms when X is an R ordered factor. This will effectively give X roughly 1.5-2.5 degrees of freedom.

Improve this answer
$\endgroup$
5
  • 3
    $\begingroup$ Thank you, Frank. I was not aware that using polynomial contrasts with lm() had problems - can you point me to some more information about why this doesn't work? I'm happy to use brms, though. I assume something like brm (Y~ mo(X)))? $\endgroup$
    – mkt
    Commented Aug 26 at 15:36
  • 1
    $\begingroup$ If you used a cubic polynomial you will fit all the points so that's effectively categorical. Yes re: brm. $\endgroup$
    – Frank Harrell
    Commented Aug 26 at 22:07
  • 1
    $\begingroup$ I would argue that the brms approach (while IMHO great) isn't the only option that respects the ordinal nature of X (e.g. monotonic splines are likely a decent option as well). Tried to make that into another answer. $\endgroup$
    – Martin Modrák
    Commented Aug 27 at 10:02
  • 1
    $\begingroup$ Thank you. I'd appreciate it if you could also address the part of the question about how to use the fitted ordinal model to take a binary decision (intervene or not). The intervention would apply to a subset of the predictor levels. If that works better as a separate question with more elaboration, I can create a new one. $\endgroup$
    – mkt
    Commented Aug 27 at 10:11
  • 2
    $\begingroup$ Think about a very simple model containing a continuous predictor such as blood pressure. How to you make a seemingly binary decision to start antihypertensive medication based on blood pressure? You estimate the risk of a bad outcome at the observed bp, with and without receiving the drug. There is no dichotomization of bp at any point. $\endgroup$
    – Frank Harrell
    Commented Aug 27 at 11:45
6
+50
$\begingroup$

To expand and complement Frank Harrell's answer:

Providing an answer to the binary question based on the full model

First, I think there is value in giving your collaborator a bit of pushback on requiring the answer to the binary question. On of the good reasons is that the binary question can be interpreted as several possible questions about the full dataset. The difference is about what assumptions do you make about the relative distribution of the "low" categories (1 vs 2 vs 3) and the relative distribution of the "high" categories (4 vs 5).

Inferences using the coefficient for the binarized predictor effectively assume that the relative frequencies in each category will be the same in the target population/future data as in the observed data. So this is a question you can answer using predictions from the full model (basically you weigh effect for each level of the full predictor by its frequency in the data). You can also get fancy and account for the uncertainty in the relative frequencies.

Another possible interpretation is to find a either a worst case or a best case estimate of a difference between the "low" and "high" categories across all possible relative frequencies within each category. This translates to either just comparing levels 3 and 4 (worst case) or levels 1 and 5 (best case) and can be answered with the full model (but not the binarized model).

Obviously, one could make other assumptions about those relative frequencies and gain somewhat different answer. Using the full model one could also get a more nuanced answer about which levels we are reasonably confident are associated with a different outcome.

If you cannot convince your collaborator to be more interested in the more nuanced answer, you should at least be able to force them to explicitly state their assumptions about the relative frequencies and answer that variant of the binary question.

Monotonic predictors

As noted by Frank, the brms package uses a neat construction to build monotonic but highly flexible predictors - details are described in Bürkner & Charpentier 2020. Although I am a fan of the Bayesian approach, there is IMHO no fundamental reason why such a construction wouldn't work in a frequentist maximum likelihood setting - I just haven't seen it done. Indeed the paper cites a couple approaches like using penalized monotonic splines (on the category index) that definitely work and are implemented in the freuqentist setting, see e.g. de Leeuw: Computing and Fitting Monotone Splines, or gamlss::pbm().

The implementations of monotonic splines I've seen require fixing the direction of the effect beforehand, but that IMHO is not a dealbraker. If the direction is unknown, you could do something like likelihood ratio test between null, increasing and decreasing model and get a decent frequentist answer.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thank you, Martin, this is helpful. I'm happy to stick with the ordinal model, but based on this model, we'd still need to take a binary decision (intervene or not). The intervention would be applied to a subset of the predictor levels. Could you comment on how best to use the ordinal model for this? If that works better as a separate question with more elaboration, I can create a new one. $\endgroup$
    – mkt
    Commented Aug 27 at 10:14
  • 2
    $\begingroup$ Very briefly: To guide decision, you also need some model of the outcome of the intervention. Presumably, the effect of the intervention depends on X. If yes, then for each of the predictor levels you can make predictions/inferences about X and translate them into predictions about the outcome. Those predictions can then be judged by whatever criteria suits the problem best (e.g. cost/benefit) and you can recommend the intervention for levels where the judgement is positive. Note that including outcomes makes this a causal inference question with all the associated difficulties. $\endgroup$
    – Martin Modrák
    Commented Aug 27 at 10:36
  • 2
    $\begingroup$ @mkt As a general rule you want to propagate the uncertainty as close to the decision point as you can, because e.g. different decisions may require different risk/benefit tradeoffs so you want the decision maker to be able to see the risk. $\endgroup$
    – Martin Modrák
    Commented Aug 27 at 10:38
  • 2
    $\begingroup$ @mkt Also note, that using the full model allows your answer to be e.g. "intervention recommended for 1 - 2, not recommended for 5, we need more data to make conclusions about levels 3 and 4". And that's obviously a good thing (although it may be some work to persuade a practitioner that "we need more data" is a good and reasonable answer :-D ) $\endgroup$
    – Martin Modrák
    Commented Aug 27 at 10:44
  • $\begingroup$ Many thanks! If you'd like to add that to your answer at some point, I think it would complete it nicely. $\endgroup$
    – mkt
    Commented Aug 28 at 10:33
-2
$\begingroup$

I will go out on a limb here (but I feel that limb is pretty strong), and say that your predictor $X$ is a Likert score. It looks llike a Likert (ordinal), it walks like a Likert (5 levels), it quacks like a Likert (poor quality of the measuring instrument).
If that is the case, then I have actually to commend your dataset owner for wanting to treat $X$ as binary, exactly for the 2 reasons you gave. I would add a 3rd one to that: there are not too many tests adapted for a strictly ordinal variable (e.g. Mann-Whitney U, or brms as suggested in another answer).
With all due respect to @Frank Harrell, in the case of a Likert score, such binning is very clearly defined, and very easily interpretable; 1-3 correspond to no expression of "agreement" (or however the 5 Likert choices were worded), while 4-5 correspond to an expression of "agreement". There is a very clear threshold; on one side disagreement or indifference, on the other "agreement".
To be honest, when dealing with Likert scores (answers to a single Likert question), I almost always deprecate them to a binomial variable. Note that this is why 4-point Likert scales are often used; to force the respondent to pick a side (and eliminate the interpretation of "neutral"). Think of this binning as similar to how Net Promoter Scores (NPS) are computed (going from an 11 points ordinal scale to a 3-points, or even to a binary scale).
So I see nothing wrong in deprecating the Likert score to a binomial (yes/no) variable. You wrote "one should not bin ordinal or continuous data". This is certainly true for continuous variables; you lose a lot -too much?- information. Note though that all dichotomous diagnostic tests (e.g. Covid, TB, pregnancy) do exactly that; take a continuous outcome, use a "properly selected" threshold, and give a binary answer. So it has its uses. When it comes to ordinal variables, specifically Likert scores, it is not even clear anymore that you would lose anything. Given the fact that a Likert score is indeed ill-defined, ambiguous, has very low resolution (5 choices), and that there are few tests which respect this ordinality, you are not losing much (if any) information by deprecating it to a binomial variable.

PS: if indeed $X$ is a Likert score, you may want to add this "detail" to your original question. It will definitively clarify your context.

Improve this answer
$\endgroup$
8
  • 4
    $\begingroup$ This is very problematic and does not represent best statistical practice. There is no threshold that will be reproducible, and a host of problems as detailed here $\endgroup$
    – Frank Harrell
    Commented Aug 26 at 22:06
  • 1
    $\begingroup$ @FrankHarrell, the link you provided is about categorizing continuous variables. My answer agrees that this is poor practice. So why the disagreement? I however note that dichotomous diagnostic tests (Covid, TB, pregnancy, etc.) all do exactly that; transform a continuous outcome (typically a level of analyte(s) of interest), and transform this into a yes/no answer; yes you are pregnant, or no you are not, based on the (in that case) level of hCG in urine (which is an analog, continuous quantity). So I agree that it is generally bad practice, but in some cases, it can serve a purpose. $\endgroup$
    – jginestet
    Commented Aug 26 at 23:35
  • 5
    $\begingroup$ I won't downvote this because you're mentioning that the procedure is bad practice, yet I have to join a chorus that throwing away some information in the data is always to be avoided if possible. $\endgroup$
    – Nick Cox
    Commented Aug 27 at 10:40
  • 4
    $\begingroup$ The link I provided was primarily about continuous variables but almost all of it also applies to ordinal variables. And thinking that the need for a yes/no answer translates into up-front dichotomization is a major mistake though a very common one. Never categorize on the front end; do so only on the back end if at all. $\endgroup$
    – Frank Harrell
    Commented Aug 27 at 11:48
  • 3
    $\begingroup$ There is almost no limit for how noisy a variable can be before dichotomization doesn't make it worse. Translating a quantitative error to a qualitative error is not a good result. $\endgroup$
    – Frank Harrell
    Commented Aug 27 at 19:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.