6
$\begingroup$

I have a regression problem with the target always nonnegative, I used gbdt as the model, but sometimes the model outputs negative prediction value.Is there any way to output nonnegative value using gbdt, or how to impose nonnegative target(prediction value) constraint for gbdt or any other machine learning model?

$\endgroup$
1
  • 3
    $\begingroup$ This depends on the software you're using. For example, in xgboost you could assume that your response is gamma-distributed and specify "objective":"reg:gamma". You can also probably enforce non-negativity in a more general way using customized objective functions. $\endgroup$
    – Sycorax
    Commented Aug 2, 2018 at 16:13

1 Answer 1

3
$\begingroup$

One way is to transform your data in such a way that negative values of your "real" variable are impossible. For instance, suppose you have a response variable Y. Define W(Y) = log(Y), then do a regression with W as your response variable. The take your predicted values, and exponentiate them to get back to Y. It's possible that your model will return a negative value of W, but e^W will always be positive.

Of course, this will raise other issues, such as heteroskedasticity and not being linear (although if you're constraining the results to be positive, then it wasn't linear to begin with). You can play around with other functions; you just need two functions $f$ and $g$ such that they are inverses of each other and one of them has a codomain of only nonnegative numbers. You should think about why you expect Y to always be positive to guide you in deciding what function to use.

$\endgroup$
3
  • $\begingroup$ In the case of gradient boosting, the linearity and heteroskedacity is not so much a problem. $\endgroup$ Commented Aug 2, 2018 at 17:52
  • 2
    $\begingroup$ I would be more concerned about the bias we introduce by the naive backtransformation, cf. Miller (1984, The American Statistician). $\endgroup$ Commented Aug 2, 2018 at 19:38
  • $\begingroup$ we take the log for response only, or for both response and predictor? thanks! $\endgroup$
    – Hao Yu
    Commented Aug 3, 2018 at 5:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.