Is it justified to discretize / bin a skewed variable in a classification problem?

Question

How would a skewed variable impact a classification problem (logistic regression, tree model)?

Is it justified to bin the skewed variable ?

My data set comprises of younger demographic and fewer older candidates which is expected since it is about college goers.

Also, since college goers are above the age of 19 could this data be considered censored ? And what impact would this have on classification ?

Note: It is important for me to understand the impact of age if any on the classification.

While this When should we discretize/bin continuous independent variables/features and when should not? advises against binning and the references talk about the Modifiable Areal Unit problem, this Why Binning Variables in Predictive Analytics? suggests that (1) There is implicit binning in decision trees and (2) Quotes an example of the impact an optimal range of temperature has on flowering which cannot be captured by continuous variable.

In my problem I am trying to learn the impact of age among other variables on graduating and most college goers are young except a few older students.

If I am building a tree model for classification, should the age be considered a skewed variable and does binning help in building a tree or is detrimental to the process of splits using entropy or gini-index ?

Wikipedia article on Binning says:

Binning is also used in machine learning to speed up[3] the decision-tree boosting method for supervised classification and regression in algorithms such as Microsoft's LightGBM and scikit-learn's Histogram-based Gradient Boosting Classification Tree

Both posts seems to agree that there is loss of information when continuous variables are binned.

Answer 1

If you are using trees then the algorithm will select the bins for you, regardless of whether the variable is skewed or normal or whatever. There is no need for you to "pre-bin" and such an approach can only make the result worse.

If you are using some kind of regression (you mention logistic regression) then you can use a spline of a continuous variable to look for nonlinearity. (There are other methods too).

So, when, if ever, should you bin? I can think of two possible reasons:

1. The cutoff of your bin is of substantive interest and perhaps part of a hypothesis. For instance, if you are interested in the behavior of people across the lifespan and have hypothesized that it has one pattern below age 18, another from 18-65 and another at older ages, one method would be to bin age in three categories. (Even here, you might use a hockey stick type model, or a spline model with knots set by you).

2. The model is too "big" to run in a reasonable time. Of course, how big is big depends on your computer and your software and what is "reasonable" is also context specific. if you are making a model that will be run only once, or only once a year, then a running time of many hours might be reasonable. If you need to run it every day, then maybe not.

One other possible reason is that binned models are easier to explain than splines. This is true, but I don't regard it as a good reason. One of your jobs, as a data analyst, is to explain things.

Finally, you might bin because it is demanded of you by your "pointed haired boss" (from Dilbert) who could be a journal editor, a dissertation advisor, a client, a boss and so on.

Answer 2

Whether to bin or not to bin may be answered by the quote (due to George Box?):

All models are wrong, but some models are useful.

Broadly, models are created to either understand the data or to make predictions (and of course for both!).

In your situation I would carry out some experiments and test a range of bin sizes starting with a no bin model.

The "no bin" model could be your baseline model.

For a prediction model your performance metric will help you assess how each bin assignment performed on some holdout data set.

For understanding the model variables you can extract their relative importance following each bin assignment to see if they "make sense" according to accepted theory.

Keep in mind that any time your model restricts your dataset in some way you will likely decrease its information content.

BUT the model may be useful!