0
$\begingroup$

Can anyone suggest forcing monotonicity in noisy data decreases the accuracy of XGBOOST?

$\endgroup$
1
2
$\begingroup$

XGBoost constructs a model by optimization. Monotonicity enforces a constraint on that model, and creates a constrained optimization problem. It's always true that constrained optimization problems have optima which are are at best as good as the unconstrained problem; if the constraint is active, then the optima must be worse than in the unconstrained problem.

Stated another way, monotonic models enforce that increasing (decreasing) the value of a feature can only increase (decrease) the value of the response. Sometimes we want to do this because we know it's physically impossible for some process to increase-then-decrease-then-increase, or we have other relevant knowledge. But if oscillation truly a part of process we're modeling, then monotonicity constraints will obviously make the model worse.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.