I attended a conference on ML and Data Science and I have a general question that was not answered in the conference.

If we have a continuous variable, let's say age. What is the best way to handle this variable. These are my thoughts, please let me know if they nonsense, but in general I think it is a very important and useful topic that has not been discussed in the detail that I need it:

  1. How should you decide on the number of bins? Would it be best to choose an arbitrary number of bins and then test various combinations, finally settling on the best fit? Should volume in bins be taken into account - for me this is important. What is the best approach to accommodate the volume and number of bins?

  2. When setting the bin width would it make sense to choose various bin widths and do hypothesis testing on the bins deciding on the boundaries based on hypothesis testing (something like a t-test) choosing boundaries once the hypothesis states that the bins are different.

  3. Is it really necessary to split the variable to start with. Specifically, some models can handle continuous variables and some models set the bins.

  4. Would it be a good idea to keep the original continuous variable along with the binned values - I am sure this is not a good idea. But I would like to know exactly why.

  • 1
    $\begingroup$ Why would you want to bin the variable? Could you provide sources? $\endgroup$ Commented Mar 1, 2017 at 8:26
  • $\begingroup$ I don't have sources. This is part of my question. But in the presentation, they binned ages less than 15 as "child", 16-60 "adult" and 60+ as "elder". So indeed, I can see that this makes logically sense. However, I have the question, should I rather have 4 bins? 5? 6? should it be up to 15, maybe 14, or 16? Should I bin at all? $\endgroup$ Commented Mar 1, 2017 at 8:30
  • 1
    $\begingroup$ There is not right or wrong way to bin. It depends on the question of interest and the binning simple is a loss of information which possibly makes a question easier. Without a specific question, there is no way to determine whether a certain binning is good or not. $\endgroup$ Commented Mar 1, 2017 at 8:54
  • $\begingroup$ @NikolasRieble thanks for the comment, so you are saying you might bon the age into 2 categories in some instances and 3 in others, depending on what your objective is. I can see it... if you are trying to model something ons specific age ranges a priori, then indeed the binning makes sense. I am talking about a case where there is not pre-determined binning required. How would you approach it in this sense? Let's assume you have the famous titanic dataset, what choices of age would you use in this instance, and why? $\endgroup$ Commented Mar 1, 2017 at 14:01
  • 2
    $\begingroup$ @CharlFrancoisMarais, I did not mean to be rude, & I apologize if I inadvertently gave offense. FWIW, my comment was not directed towards you. Your question was being voted to be closed as unclear, & I was arguing that your question should be considered acceptable here. Please refrain from ad hominems, even if you think someone has been rude. $\endgroup$ Commented Mar 1, 2017 at 14:42

2 Answers 2

  1. The number of bins is individual decision. With age, you might have natural intervals that makes sense when interpreting results. Let’s say you have a dataset of people from 10 to 80 years. It makes more sense to make bins for children, adolescence, adult and old people, rather than create 8 bins of 10 years. There are several approaches I could think of. You can create bins based on quantiles or just use equal intervals. You should always see how they affect your model. I normally start (if possible) with original continuous variable and try several possibilities.

  2. You might do previous analysis to determine what are natural bins in your case. But otherwise you will see if the variable will be significant in the model.

  3. No. I would always start with continuous.

  4. In the dataset, yes. In the model there would be strong colinearity between original and binned variable. I think it is worth to try using binned age as level 2 variable and original age as level 1 variable in some cases.

  • $\begingroup$ Thanks for the comments Martin, much appreciated! Regarding point 1, I fully agree with you that it should be natural and depends on the situation. But here is the thing, sure it will be natural to group child, adult and elder. BUT it could also make sense to group enfant/teenager/young adult/adult/elder. How do you know when to stop and start. Attached with this, you can say a young adult is 19-24 or 18-25/26, or any other bindings.So it leaves a lot of questions open for interpretation. $\endgroup$ Commented Mar 1, 2017 at 9:32
  • $\begingroup$ For 3, if you start with continuous, then how do you proceed from this to a grouped vision? $\endgroup$ Commented Mar 1, 2017 at 9:34
  • $\begingroup$ for 4, I agree, this is what I was afraid of as well. What do you mean with level 1 and 2 cases? $\endgroup$ Commented Mar 1, 2017 at 9:36

Disclaimer: this answer possibly belongs in the comments, but I don't have enough reputation points to post comments.

1,2. To help you answer questions 1. and 2. yourself, I have two pointers for you.

First is this paper: A Survey of Discretization Techniques: Taxonomy and Empirical Analysis in Supervised Learning. Of course, which exact method to use depends on your use case; as the authors note in the conclusion section:

"A researcher/practitioner interested in applying a discretization method should be aware of the properties that define them in order to choose the most appropriate in each case. The taxonomy developed and the empirical study can help to make this decision."

Second is the Bayesian Blocks method, which you can learn more about from this 2012 blog post, which heavily relies on this paper. As noted in the blog:

"The adaptive-width bins lead to a very clean representation of the important features in the data. More importantly, these bins are quantifiably optimal, and their properties can be used to make quantitative statistical statements about the nature of the data."

(For a pre-built example in Python, you can refer to its implementation in the scikit-hep project.)

  1. It's not necessary to do so if you have methods that deal with continuous variables and produce good results for your problem. However, sometimes binning may be the best/most practical approach (for example when you want to learn a Bayesian network from your data), in which case you can try one of the methods from the links I shared above.

  2. As already pointed out in a previous answer, this would probably not be a good idea due to the correlation/redundancy that will be present between the original data and what's derived from it by binning.

  • $\begingroup$ This space is for answers not long comments. It doesn't matter whether or not you have enough reputation to make a comment. You should wait until you do. $\endgroup$ Commented Aug 16, 2019 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.