I have heard people say, "One of the disadvantages of neural networks is that they are generally less interpretable". But I wonder, how is another model, such as XGBoost, more interpretable than neural networks? In XGBoost, one could use feature importance, SHAP, or PDP to explain the model, which I believe can also apply to neural networks?
2
-
1$\begingroup$ It probably depends on the context in which you found that quote, and in particular what "traditional ML" is. Neural networks have been around far longer than XGBoost, so I don't quite see why XGBoost would be "more traditional" than NNs. It's quite possible that for your quote, "traditional ML" refers to regression, GAMs etc. $\endgroup$– Stephan KolassaMar 1 at 7:44
-
$\begingroup$ Thanks for the feedback. I now see the confusion and have updated the question, thanks! $\endgroup$– J21Mar 1 at 7:51
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Both (sufficiently large) neural networks and XGBoost are not interpretable on their own (they are not "intrinsically interpretable") and are typically seen as part of the same "not interpretable" category. Both can be interpreted post-hoc using various methods such as feature importance, SHAP, PDP, saliency maps, counterfactual explanations, etc.
As Stephan Kolassa already mentioned, the "more interpretable" traditional models neural networks are often compared with are much simpler models like logistic regression or simple (non-boosted, non-ensemble) decision trees. (These often perform surprisingly well when compared with modern neural networks in a proper evaluation. Even in image processing, logistic regression based on interpretable hand-crafted features can very well outperform modern CNNs.)
In a separate branch of research, there is a push to make neural networks intrinsically interpretable.
-
$\begingroup$ This answer is delightful. It's my honor to receive such a high-quality answer. Thanks, Eike! $\endgroup$– J21Mar 1 at 21:32
-