Choosing to keep or bin continuous variables for communicating regression results

Question

I am using logistic regression in a work setting (e.g. subscription conversion in a technology product). I would like to communicate about what independent variables a team should focus on to increase an outcome variable.

The independent variables of this regression include both continuous and categorical variables. Let’s say we have these independent variables:

• is_cat (Categorical)

• Day_active (Continuous)

• Avg_items (Continuous)

If we run the regression with these variables “as is”, communicating the variable coefficients relative to each other could be confusing. Increasing Avg_items by 1 may have a much smaller impact than increasing day_active by 1, but increasing Avg_items by X may lead to the same impact as increasing day_active by 1. This insight is somewhat hidden though since a stakeholder just looking at the coefficients might conclude increasing day_active by 1 has a higher ROI than focusing on Avg_items, even though that may not be true.

Also, it's non-intuitive comparing the coefficients of a categorical and continuous variable, especially for prioritization.

Another choice is to change Day_active and Avg_items into categorical variables so that the variables become unitless and therefore coefficients more readily comparable.

I’ve seen people transform continuous variables into categorical ones by looking at the distribution of the variable and splitting it at the knee in the distribution or just using quantiles, etc. Seems reasonable.

The obvious downside is that we could be losing a lot of information binning the continuous variables into categorical ones. I guess we could compare regression diagnostics between a model with the continuous variables and one with just categorical variables to quantify how much we are losing. But the loss may be worth it if it makes the results easier to explain and understandable by stakeholders.

Does this all make sense? Or are there other ideas I should be considering for communicating drivers of a product outcome using regression, or communicating headroom?

Answer 1

+1 to Demetri's answer.

Skip model coefficients altogether. (Better: put them in an appendix, in case someone asks for them, and to show you did your homework.) As you write, they are hard to interpret and can be gotten wrong. But more importantly, nobody cares about your coefficients, anyway! What people care about is the impact on the observable.

So pick a few reasonable scenarios and twiddle your predictors in a meaningful (!) way. For instance, take an average user, or a modal one, and report what would be predicted happen to this user should Day_active or Avg_items be changed in a meaningful way. If your Days_active typically spans months or years, then a change by a single day will not mean much - but a change by one month or three months will mean something. If your Avg_items is in the low singletons, then increasing this by a single unit will be a meaningful change.

Also keep in mind how easy it is to affect these predictors. If it is equally easy or expensive to change Day_active by 30 days or Avg_items by 1 unit, then these changes are comparable and meaningful.

Note how the initial step of picking a scenario already amounts to a kind of binning. And that is fine in this context. Other scenarios to pick would be users on the high end of the observable (can we actually influence high-value users any more, or are they already at a "ceiling"?), or users on the low end (likely a "long tail" that only interacted once - there will likely be a large number of such users, is it worthwhile to try to reactivate them?).

Yes, this means that you need to consider the wider picture, not just your model, but also the business implications and costs or your report, and what baseline scenarios are "interesting" to look at. And in my opinion, that is how it should be. Part of our job as statisticians or data scientists is to help other people understand our results. For that, we need to understand their language and take much of the translation task on our shoulders.

Answer 2

Dichotomozing and binning continuous variables is a poor poor idea. Not only are the splits more or less arbitrary, but splitting continuous variables leads to residual confounding and decreases the degrees of freedom in the model (maybe not a problem in your particular example, you likely have oodles of data). This can lead to poorer measures of model performance (like calibration).

Additionally, splitting the predictors can fool the analyst into believing noise is signal. Here is an example. Below, I've simulated binary data and a continuous predictor. I've binned the predictor into 5 bins and computed the means in each bin. When I simulated that data, I made sure that as the predictor increased then so too would $E(y \vert x)$. However, when I plot the binned data, I see a different story. It would appear the data have a non-linear effect -- something that would fail to generalize to new data.

You seem to be vexed by apparent confusion in the interpretation of model coefficients. Without sounding flippant, I would assume it is part of your job to explain to stakeholders what these coefficients mean, rather than leaving them to do it. If the insight you mention is hidden, it might be worthwhile to elucidate it.

Alternatively, you can eschew model coefficients completely as their effect is somewhat obfuscated and depends on the baseline risk. A log odds ratio of 2.0 be associated with a massive risk difference if the comparitor's risk is moderate, or it can be associated with a paltry risk difference if the outcome is certain (near 0 or 1). Consequently, I usually report probabilities of various scenarios to try to communicate how interventions might affect risk.

For example, I might say something like "If customers had one more item in their basket on average, we might expect a p% change in probability of sign up from x% to x+p%". This might even be a good opportunity to do some visualizing. Because you can not realistically intervene and change predictors in an enormous way, I find this approach sufficient.

All in all, needlessly categorizing hurts rather than helps. If you're having trouble explaining model coefficients, my advice would be to take a different approach and report model probability estimates under various scenarios. Skip coefficients entirely.