5
$\begingroup$

I would like some insight on known approaches for linear extrapolation on tails of xgboost models. The current model is missing data at the distribution tails and is thus predicting flat trends for certain variables when in reality these trends are (known to be) continuing from previous parts of the model with enough data.

I've ran a few simulations on test data to give me a rough idea of how the model will shape at that particular variables inputs and would like to perform a extrapolation to help the model where it begins to flatten off.

Is there a robust way of performing such an extrapolation without simply picking points at random and creating linear inference? Any help would be greatly appreciated!


Edit: Just for clarity, the simulation is run about 10 times on the test data and outputs similar looking graphs in which I would ideally like to use the data from all the graphs (weighted equally to get the best idea) for the extrapolation.

sample graph(sample graph where data is missing for lower values but the true nature of the trend follows the linear plotting afterward. i.e. goal is to extrapolate a trend on this flat part of the graph using all the 10 output models)

$\endgroup$
3
  • $\begingroup$ Adding an image of what you mean could improve the answers. $\endgroup$
    – Michael M
    Commented Jan 18 at 20:33
  • 1
    $\begingroup$ @MichaelM Added! $\endgroup$
    – aort01
    Commented Jan 18 at 21:49
  • $\begingroup$ @wjktrs Touching the logitisics of the model is not an option, extrapolation is the only goal here $\endgroup$
    – aort01
    Commented Jan 18 at 21:50

1 Answer 1

2
$\begingroup$

In general, tree-based models are not exceptional interpolators because effectively they make a recursive space partitioning with step functions of constant value. An option would be to use LightGBM's linear_tree option where the "leaves have linear models". I would probably though use first a GAM instead of GBM if extrapolation was my main target.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.