4
$\begingroup$

Lot of discussion in CrossValidated focuses on optimal binning methods, binning example etc. But I am trying to figure out what are the scenarios that I have to bin variables whereas it's better idea to treat explaining variables as continuous then binning in predictive modeling. Any rule of thumb for me to follow? Thanks!

$\endgroup$
3
2
$\begingroup$

There are a lot more options for Classification techniques in ML literature compared to analysis for continuous outcomes. Models like Regression trees, J4.8 implicitly create bins on variables and create the tree on the lines of a regular decision tree.

The second reason is deviation from normality in terms of skewness and multi-modal nature of univariate distributions. For instance, if you want to understand the impact of temperature on the flowering of a plant, there would be an optimal range of temperature. If you model for temperature as a continuous variable, it may not capture the influence in the right manner. A better approach is to account for high-order effects of temperature in the model. A third alternative may be to bin the variable into low, medium, high levels (discretize/bin it). You could always increase the resolution by increasing the size of bins. A down-side of binning is the loss of information due to discretization in many cases.

Quoting from this book:

The intervals the variables will be discretized into can be chosen in one of the following ways: - Using prior knowledge on the data. The boundaries of the intervals are defined, for each variable, to correspond to significantly different real-world scenarios, such as the concentration of a particular pollutant (absent, dangerous, lethal) or age classes (child, adult, elderly).

  • Using heuristics before learning the structure of the network. Some examples are Sturges, Freedman-Diaconis, or Scott rules (Venables and Ripley, 2002).
  • Choosing the number of intervals and their boundaries to balance accuracy and information loss (Kohavi and Sahami, 1996), again one variable at a time and before the network structure has been learned. A similar approach considering pairs of variables is presented in Hartemink (2001).
  • Performing learning and discretization iteratively until no improvement is made (Friedman and Goldszmidt, 1996). These strategies represent different trade-offs between the accuracy of the discrete representation of the original data and the computational efficiency of the transformation.
$\endgroup$
2
  • 2
    $\begingroup$ Much of this is useful information but some of it appears to confuse important concepts, particularly the distinction between the symmetry of the distribution of a variable and linearity of a relationship between two variables. It also seems incomplete by not discussing the disadvantages of binning. In particular, you cannot always increase the resolution because there is a limit to the available degrees of freedom. This is of concern because the OP is asking for guidance (a "rule of thumb") about when and how to bin variables. $\endgroup$ – whuber May 14 '15 at 19:12
  • $\begingroup$ @whuber, you are right. These are important points that I need to consider. I will chart a better answer for rule of thumb. I realized that based on your inputs that I need to explicate what I mean by binning for multimodal inputs. As you mentioned, it may confuse readers and they may resort to binning multimodal distributed variables always which shouldnt be the case. $\endgroup$ – KarthikS May 20 '15 at 0:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.