Treating continuous variables

Question

I attended a conference on ML and Data Science and I have a general question that was not answered in the conference.

If we have a continuous variable, let's say age. What is the best way to handle this variable. These are my thoughts, please let me know if they nonsense, but in general I think it is a very important and useful topic that has not been discussed in the detail that I need it:

1. How should you decide on the number of bins? Would it be best to choose an arbitrary number of bins and then test various combinations, finally settling on the best fit? Should volume in bins be taken into account - for me this is important. What is the best approach to accommodate the volume and number of bins?

2. When setting the bin width would it make sense to choose various bin widths and do hypothesis testing on the bins deciding on the boundaries based on hypothesis testing (something like a t-test) choosing boundaries once the hypothesis states that the bins are different.

3. Is it really necessary to split the variable to start with. Specifically, some models can handle continuous variables and some models set the bins.

4. Would it be a good idea to keep the original continuous variable along with the binned values - I am sure this is not a good idea. But I would like to know exactly why.

Answer 1

1. The number of bins is individual decision. With age, you might have natural intervals that makes sense when interpreting results. Let’s say you have a dataset of people from 10 to 80 years. It makes more sense to make bins for children, adolescence, adult and old people, rather than create 8 bins of 10 years. There are several approaches I could think of. You can create bins based on quantiles or just use equal intervals. You should always see how they affect your model. I normally start (if possible) with original continuous variable and try several possibilities.

2. You might do previous analysis to determine what are natural bins in your case. But otherwise you will see if the variable will be significant in the model.

3. No. I would always start with continuous.

4. In the dataset, yes. In the model there would be strong colinearity between original and binned variable. I think it is worth to try using binned age as level 2 variable and original age as level 1 variable in some cases.

Answer 2

Disclaimer: this answer possibly belongs in the comments, but I don't have enough reputation points to post comments.

1,2. To help you answer questions 1. and 2. yourself, I have two pointers for you.

First is this paper: A Survey of Discretization Techniques: Taxonomy and Empirical Analysis in Supervised Learning. Of course, which exact method to use depends on your use case; as the authors note in the conclusion section:

"A researcher/practitioner interested in applying a discretization method should be aware of the properties that define them in order to choose the most appropriate in each case. The taxonomy developed and the empirical study can help to make this decision."

Second is the Bayesian Blocks method, which you can learn more about from this 2012 blog post, which heavily relies on this paper. As noted in the blog:

"The adaptive-width bins lead to a very clean representation of the important features in the data. More importantly, these bins are quantifiably optimal, and their properties can be used to make quantitative statistical statements about the nature of the data."

(For a pre-built example in Python, you can refer to its implementation in the scikit-hep project.)

3. It's not necessary to do so if you have methods that deal with continuous variables and produce good results for your problem. However, sometimes binning may be the best/most practical approach (for example when you want to learn a Bayesian network from your data), in which case you can try one of the methods from the links I shared above.

4. As already pointed out in a previous answer, this would probably not be a good idea due to the correlation/redundancy that will be present between the original data and what's derived from it by binning.