10
$\begingroup$

So I've read a few posts about why binning should always be avoided. A popular reference for that claim being this link.

The main getaway being that the binning points (or cutpoints) are rather arbitrary as well as the resulting loss of information, and that splines should be preferred.

However, I am currently working with the Spotify API, which has a bunch of continous confidence measures for several of their features.

Looking at one feature, "instrumentalness", the references state:

Predicts whether a track contains no vocals. “Ooh” and “aah” sounds are treated as instrumental in this context. Rap or spoken word tracks are clearly “vocal”. The closer the instrumentalness value is to 1.0, the greater likelihood the track contains no vocal content. Values above 0.5 are intended to represent instrumental tracks, but confidence is higher as the value approaches 1.0.

Given the very left-skewed distribution of my data (about 90% of the samples are barely above 0, I found it sensible to transform this feature into two categorical features: "instrumental" (all samples with a value above 0.5) and "non_instrumental" (for all samples with a value below 0.5).

Is this wrong? And what would have been the alternative, when nearly all of my (continous) data is revolving around a single value? From what I understand about splines, they would not work with classification problems (what I'm doing) either.

$\endgroup$
  • 10
    $\begingroup$ The setup you describe doesn't seem to be to imply that binning is a good idea. You said it yourself that there is information in how close to 1.0 a value is. IMHO you'd do well to have a continuous feature that is related to the probability of being instrumental. Perhaps you can expand on your question. $\endgroup$ – Frank Harrell Feb 4 at 13:21
  • $\begingroup$ My question basically is when it is okay to use binning, if at all. In my case, I used it on the basis of the domain (instrumental/not instrumental), as I believe it to be more predicative than saying how close a track is to being instrumental (since a track either is or is not instrumental). You argued against this logic however in point 8 of your post. I, as a novice, just have a hard time really understanding as to why that should be. $\endgroup$ – Readler Feb 4 at 13:34
  • 1
    $\begingroup$ I wrote a long post about this in the context of predictive modeling: madrury.github.io/jekyll/update/statistics/2017/08/04/… $\endgroup$ – Matthew Drury Feb 4 at 16:31
  • $\begingroup$ Very informative and thorough, thanks. However, I don't see the relation to my question (though I still gained some new insights, so all's well!). Your article is talking about binning the predictor variable in regression problems and why that is a bad idea (which your article convincingly argued against) and why using splines helps for modelling regression. I was asking about why it's bad to discretise the values of a continous feature (an input) in a classification problem (whose predictor variables are inherently "bins", i.e. classes). $\endgroup$ – Readler Feb 4 at 16:43
  • 2
    $\begingroup$ If nearly all your feature is at one point, then it's likely to be unhelpful to your model, regardless of what you do. $\endgroup$ – Acccumulation Feb 4 at 18:29
15
$\begingroup$

It is a slight exaggeration to say that binning should be avoided at all costs, but it is certainly the case that binning introduces bin choices that introduce some arbitrariness to the analysis. With modern statistical methods it is generally not necessary to engage in binning, since anything that can be done on discretized "binned" data can generally be done on the underlying continuous values.

The most common use of "binning" in statistics is in the construction of histograms. Histograms are similar to the general class of kernel density estimators (KDEs), insofar as they involve aggregation of step functions on the chosen bins, whereas the KDE involves aggregation of smoother kernels. The step function used in a histogram is not a smooth function, and it is generally the case that better kernel functions can be chosen that are less arbitrary under the KDE method, which also yield better estimates of the underlying density of the data. I often tell students that a histogram is just a "poor man's KDE". Personally, I would never use one, because it is so easy to get a KDE without binning the data, and this gives superior results without an arbitrary binning choice.

Another common use of "binning" occurs when an analyst wishes to discretize continuous data into bins in order to use analytical techniques that use discrete values. This appears to be what is being suggested in the section you quote regarding prediction of vocal sounds. In such cases there is some arbitrariness introduced by the binning and there is also a loss of information. It is again best to avoid this if possible, by trying to form a model directly on the underlying continuous values, rather than forming a model on the discretized "binned" values.

As a general rule, it is desirable for statisticians to avoid analytical techniques that introduce arbitrary assumptions, particularly in cases where alternative techniques are available to easily avoid these assumptions. So I agree with the sentiment that binning is generally unnecessary. It certainly should not be avoided at all costs since costs are important, but it should generally be avoided when there are simple alternative techniques that allow it to be avoided without any serious inconvenience.

$\endgroup$
  • $\begingroup$ I see. Follow up question, though: looking at the distribution of the example mentioned above see here (ironically a histogram), I just fail to see the usefulnes in a continous variable where nearly all samples revolve around one value (here being 0), which is was what initially led me to binning this feature. You mentioned alternativess - would you kindly elaborate or point me to the right direction as to where I could learn more? $\endgroup$ – Readler Feb 4 at 13:40
  • $\begingroup$ Try reading about KDEs and also consider some alternative ways of plotting univariate data. $\endgroup$ – Reinstate Monica Feb 4 at 14:07
  • $\begingroup$ I that histogram I see values all over the place (but, yes, mostly close to zero). There should be no inconvenience in using a spline fit, and that will certainly give more information. Plot the fitted spline! and , if for some reason you must discretice, that plot could help you in how. It might just be, that for your particular use, another cutpoint than 0.5 is better. $\endgroup$ – kjetil b halvorsen Feb 4 at 14:16
  • 2
    $\begingroup$ A histogram cannot be correctly construed as a KDE. What would the kernel be? $\endgroup$ – whuber Feb 4 at 14:21
  • 1
    $\begingroup$ With regards to your third paragraph, I had a similar question arise when I was trying to calculate information gain with some numeric data. Can you look at this question and explain what to do in this situation?stats.stackexchange.com/questions/384684/… $\endgroup$ – astel Feb 4 at 22:29
4
$\begingroup$

I would normally argue strongly against categorisation of continuous variables for the reasons well expressed by others notable Frank Harrell. In this case it might be helpful though to ask oneself about the process which generated the scores. It looks as though most of the scores are effectively zero perhaps with some noise added. A few of them are rather close to unity again with noise. Very few lie in between. In this case there seems rather more justification for categorising since one could argue that modulo the noise this is a binary variable. If one does fit it as a continuous variable the coefficients would have meaning in terms of change in the predictor variable but in this case over most of its range the variable is very sparsely populated so that seems unattractive.

$\endgroup$
  • 4
    $\begingroup$ My short answer to when binning is OK to use is this: When the points of discontinuity are already known before looking at the data (these are the bin endpoints) and if it is known that the relationship between x and y within each bin that has non-zero length is flat. $\endgroup$ – Frank Harrell Feb 4 at 16:49
2
$\begingroup$

Imagine you have a watch that shows only the hours. By only I mean that it has only the hour arrow that once an hour makes a 1/12 jump to another hour, it does not move smoothly. Such clock wouldn't be very useful, since you wouldn't know if it is five past two, half past two, or ten to three. That's the problem with binned data, it loses details and introduces the "jumpy" changes.

$\endgroup$
  • 1
    $\begingroup$ (+1) Yes, and add to that the additional problem that the watch-maker might not choose hourly increments, but might arbitrarily decide that his watch will be in 19 minute increments, and you have an additional problem beyond just the loss of information. $\endgroup$ – Reinstate Monica Feb 5 at 12:49
2
$\begingroup$

For some applications, apparently including the one which you are contemplating, binning can be strictly necessary. Obviously to perform a categorization problem, at some point you must withdraw categorical data from your model, and unless your inputs are all categorical as well, you will need to perform binning. Consider an example:

A sophisticated AI is playing poker. It has evaluated the likelihood of its hand being superior to the hands of other players as 70%. It is its turn to bet, however it has been told that it should avoid binning at all costs, and consequently never places a bet; it folds by default.

However, what you have heard may well be true, in that prematurely binning of intermediate values surrenders information that could have been preserved. If the ultimate purpose of your project is to determine whether you will "like" the song in question, which may be determined by two factors: "instrumentalness" and "rockitude", you would likely do better to retain those as continuous variables until you need to pull out "likingness" as a categorical variable.

$$ \mathrm{like} = \begin{cases} 0 & \mathrm{rockitude} * 3 + \mathrm{instrumentalness} * 2 < 3 \\ 1 & \mathrm{rockitude} * 3 + \mathrm{instrumentalness} * 2 \ge 3 \end{cases} $$

or whatever coefficients you deem most appropriate, or whatever other model appropriately fits your training set.

If instead you decide whether something is "instrumental" (true or false) and "rocks" (true or false), then you have your 4 categories laid out before you plain as day:

  1. instrumental, rocks
  2. non-instrumental, rocks
  3. instrumental, no rocks
  4. non-instrumental, no rocks

But then all you get to decide is which of those 4 categories you "like". You have surrendered flexibility in your final decision.

The decision to bin or not to bin depends entirely on your goal. Good luck.

$\endgroup$
2
$\begingroup$

If you bin, every result $R$ you report will be conditional on the set of bins you use. It is then up to you to average over these choices before you report any robust result. If you are ambitious (or if a reviewer gives you no choice), you may report the distribution of your results P(R) over the set of bin selection.

More details: a result $R$ is obtained from a numerical experiment in which binning was used. Let the binning be defined as $\{b_1 \cdots b_N\}$ where $b_i=[l_i,u_i]$ is the choice of $l_i$ as the lower bound and $u_i$ as the upper bound for the $i$th bin.

For simplicity, let's say the set of bins is defined by the position $l=l_0$ of the first bin and a uniform width $w$ for every bin. The lower bound of the first bin $l_0$ can vary up the upper value of the first bin $u_0=l_0+w$ and $w$ can vary from some minimum to maximum values $(w_{min},w_{max})$. To show robustness of R, we need to calculate

$$ P(R) = \sum_{w=w_{min}}^{w_{max}}\sum_{l=l_0}^{l_0+w} P(R|l,w) P(l,w) \\ P(l,w) \sim \frac{2(u_0-l_0)}{w_{max}+w_{min}} \times (w_{max}-w_{min}) $$

Of course, now you've introduced $w_{max}, w_{min},$ and $l_0$, so technically $P(R) \rightarrow P(R|w_{max}, w_{min},l_0)$, but if we suspect (not unreasonably) that $P(R)$ is independent of these values, then $P(R|w_{max}, w_{min},l_0)=P(R)$ (whew!) which is usually the case, and you rarely have to prove that unless you are really very unlucky with your reviewer!

In the context of the OP's question I would be satisfied if the arbitrary threshold 0.5 were set to a variety of values between credible min and max values, and to see that the basic results of his analysis are largely independent of the selection.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.