3
$\begingroup$

I'm seeking recommendations for explainable regression models for tabular data. I'm looking for approaches that offer a balance between complexity and interpretability – something more sophisticated than simple linear regression or decision trees.

While I'm aware of popular options like SHAP, LIME, and TabNet, I'm interested in exploring other state-of-the-art methods.

Key requirements include:

  • High Accuracy: Performance should be comparable to more complex models.
  • Explainable Predictions: The model should offer insights into how it makes predictions.

Could you please suggest some model names that fit these criteria?

Thank you!

$\endgroup$
2
  • $\begingroup$ Welcome to Cross Validated! How complex do you want to let models become? After all, (generalized) linear models allow for nonlinear basis functions that can be arbitrarily complex. There is even an argument that deep learning is linear regression with polynomial basis functions. Does that count as an "interpretable" regression model? If so, why isn't the deep neural net? $\endgroup$
    – Dave
    Apr 18 at 15:59
  • $\begingroup$ If functional complexity achieves great performance advantages over a function that is mostly additive and monotonic or even linear in its predictions, I wonder how it could be explainable (which, if you have many variables, means their contribution to the predictions are mostly additive and monotonic) $\endgroup$
    – CloseToC
    Apr 18 at 16:03

1 Answer 1

9
$\begingroup$

My book and course notes are all about this topic. The grand design is to use subject matter knowledge to formulate a model that is likely to fit the data, make the model as flexible as the effective sample size will support, use the model no matter what its complexity, and interpret the model simply using graphical methods such as partial effects plots and nomograms. These interpretations are easy to do because they are based on computing predicted values and difference in predicted values from the model.

Don't get involved in the parsimony vs. complexity debate. Think about accuracy and what's likely to fit. Then learn about avoiding overfitting.

$\endgroup$
2
  • 1
    $\begingroup$ Suppose the best fitting (out of sample) model is an extremely non additive function of 50 variables. Can we somehow understand why this model fits the data so well based on partial effects, which average out the dependency on other predictors when quantifying how the prediction changes when one predictor is varied? $\endgroup$
    – CloseToC
    Apr 18 at 16:07
  • $\begingroup$ Please state your needs in more details. Also consider relative explained variation (REV) along with confidence intervals for it. Doing variable clustering to learn how the predictors correlate then computing REV on groups of variables may also help. $\endgroup$ Apr 19 at 11:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.