Why Binning Variables in Predictive Analytics?

Question

Lot of discussion in CrossValidated focuses on optimal binning methods, binning example etc. But I am trying to figure out what are the scenarios that I have to bin variables whereas it's better idea to treat explaining variables as continuous then binning in predictive modeling. Any rule of thumb for me to follow? Thanks!

Answer 1

There are a lot more options for Classification techniques in ML literature compared to analysis for continuous outcomes. Models like Regression trees, J4.8 implicitly create bins on variables and create the tree on the lines of a regular decision tree.

The second reason is deviation from normality in terms of skewness and multi-modal nature of univariate distributions. For instance, if you want to understand the impact of temperature on the flowering of a plant, there would be an optimal range of temperature. If you model for temperature as a continuous variable, it may not capture the influence in the right manner. A better approach is to account for high-order effects of temperature in the model. A third alternative may be to bin the variable into low, medium, high levels (discretize/bin it). You could always increase the resolution by increasing the size of bins. A down-side of binning is the loss of information due to discretization in many cases.

Quoting from this book:

The intervals the variables will be discretized into can be chosen in one of the following ways: - Using prior knowledge on the data. The boundaries of the intervals are defined, for each variable, to correspond to significantly different real-world scenarios, such as the concentration of a particular pollutant (absent, dangerous, lethal) or age classes (child, adult, elderly).

• Using heuristics before learning the structure of the network. Some examples are Sturges, Freedman-Diaconis, or Scott rules (Venables and Ripley, 2002).
• Choosing the number of intervals and their boundaries to balance accuracy and information loss (Kohavi and Sahami, 1996), again one variable at a time and before the network structure has been learned. A similar approach considering pairs of variables is presented in Hartemink (2001).
• Performing learning and discretization iteratively until no improvement is made (Friedman and Goldszmidt, 1996). These strategies represent different trade-offs between the accuracy of the discrete representation of the original data and the computational efficiency of the transformation.