2
$\begingroup$

I am planning to use XGBoost to fit spatio temporal dataset that I made into a column row format. My goal isn’t prediction, but rather a regression with good fit.

I just want a model fit that’s good enough after which I can use it to learn insights from my data. Maybe by using partial dependence or SHAP. My hypothesis is that a model that’s fitted well can capture the “physics” of the system enough after which I can try exploring that physics. I won't go into train-tests split, but rather planning to fit the model into the whole dataset.

My reason for using a boosted tree algorithm is the data itself. I believe that the features have certain quantiles which influence the target, so I believe a decision tree can capture that.

Do you think this is plausible?

$\endgroup$

1 Answer 1

2
$\begingroup$

If you want "insights" you should use a model that is interpretable by itself. When you fit XGBoost, it learns to approximate the distribution of the data. When you use SHAP to explain the results, it is a black box, it approximates the results obtained by XGBoost. So you end up with two layers of approximation. Also, SHAP provides only the local explanations of the individual predictions, by itself it won't pinpoint the global "physics of the system". So there comes a third layer of approximation, where you would need to interpret the results of SHAP and derive the global explanations from them. Moreover, the whole procedure does nothing to prevent overfitting, so the scenario where your model learns to mimic the training data but is not able at all to generalize outside it.

Why not use something like good, old linear regression? It is directly interpretable, so it avoids most of the problems mentioned above.

$\endgroup$
2
  • 1
    $\begingroup$ Also be warned! Shap values (and any other model explainers) can arbitrarily mislead you, if you have confounders: you can easily run into the Simpson Paradoxon en.m.wikipedia.org/wiki/Simpson%27s_paradox , see image on Wikipedia with the different slopes in the regression setting $\endgroup$
    – Ggjj11
    Jul 19, 2023 at 9:31
  • $\begingroup$ Thanks for the answer. I am using linear regression too, but the system I am studying is highly non-linear and with a lot of features that can influence it. That's why I want to use something like XGBoost to approximate it. I know that SHAP values aren't final, but with the lack of methods that exists, I think SHAP is the most consistent of all? $\endgroup$ Jul 19, 2023 at 13:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.