2
$\begingroup$

I read that averaging and binning a continuous predictor variable is in general a bad idea because it's always better to fit the continuous relationship through splines, poly and all of that. Sure, I agree, especially for smaller, accurately measured data sets.

But what about big data and exponential distributions, where noise is more frequent and we don't necessarily want to skew the coefficients towards the center of the distribution, where we have most of the observations (although less interesting for our analysis)? Doesn't binning the predictors and the response variable reduce noise and improve our analysis for the full distribution?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ What does "noise is more frequent" mean? Isn't there noise in every observation? How do you get more frequent than that? What exactly is it you're trying to achieve? $\endgroup$
    – Glen_b
    Commented Sep 17, 2014 at 1:31
  • $\begingroup$ See this question $\endgroup$
    – Scortchi ♦
    Commented Nov 18, 2014 at 13:47

1 Answer 1

Reset to default
2
$\begingroup$

How would binning a variable "reduce noise"? It seems to me, whatever sort of measurement error your variables have, binning is always adding additional measurement error in your variables. I'm particularly skeptical of binning your outcome variable.

That said, I don't oppose binning in all circumstances. Binning predictors is sometimes an effective way to model non-linear relationships. It's primary advantage over other approaches is that the coefficients are easily interpretable. Sometimes that sort of ease of interpretation is a high priority.

Regardless of whether you choose to bin or not, I urge you to be careful to avoid overfitting. You want a better fit, not an overfit.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Can elaborate a bit more about "binning predictors is an effective way to model non-linear relationships"? $\endgroup$
    – Robert Kubrick
    Commented Sep 16, 2014 at 17:31
  • 1
    $\begingroup$ You misquote me, I said "sometimes an effective..." ;) But what I meant by that in comparison to a model with a continuous variable entered linearly, a binned version of that variable with multiple bins may lead to a model with better predictive performance. Predictive performance with a binned variable may be similar to a model with quadratic and cubic terms or a spline. I never bin outcome variables, and I only bin predictors in a regression when the underlying relationship is non-linear and I want coefficients that are easier to interpret than a poly/spline would be. $\endgroup$
    – Michael Bishop
    Commented Sep 16, 2014 at 22:05
  • $\begingroup$ Ok. I don't see why binning a predictor would improve the non-linear relationship. I do see that in my $R^2$ but I can't get the intuition... $\endgroup$
    – Robert Kubrick
    Commented Sep 16, 2014 at 23:27
  • 1
    $\begingroup$ Well, if the true relationship is non-linear, then a binned variable w/ > 2 bins will result in predictions that are non-linearly related to the predictor resulting in a better fit... it's the same reason adding polynomial terms may improve the fit. Try this: fit two models 1) with the continuous predictor and save the residuals, 2) with the binned predictor. Then plot a boxplot of the residuals grouped by the binned values of the predictor. The residuals from the binned model will be more closely centered around zero. $\endgroup$
    – Michael Bishop
    Commented Sep 17, 2014 at 2:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.