Can anyone suggest forcing monotonicity in noisy data decreases the accuracy of XGBOOST?
1
-
$\begingroup$ Hi, welcome. Please include relevant hyperlink(s), e.g. to monotonicity constraints (this perhaps: xgboost.readthedocs.io/en/latest/tutorials/monotonic.html). Further, expand on what your exact issue is. Thank you – Reviewer $\endgroup$– JimCommented Jul 16, 2018 at 7:31
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
XGBoost constructs a model by optimization. Monotonicity enforces a constraint on that model, and creates a constrained optimization problem. It's always true that constrained optimization problems have optima which are are at best as good as the unconstrained problem; if the constraint is active, then the optima must be worse than in the unconstrained problem.
Stated another way, monotonic models enforce that increasing (decreasing) the value of a feature can only increase (decrease) the value of the response. Sometimes we want to do this because we know it's physically impossible for some process to increase-then-decrease-then-increase, or we have other relevant knowledge. But if oscillation truly a part of process we're modeling, then monotonicity constraints will obviously make the model worse.
