Machine learning texts describing algorithms such as gradient boosting machines or neural networks often comment that these models are good at prediction, but this comes at the price of a loss of explainability or interpretability. Conversely, single decision trees and classical regression models are labelled as good at explanation, but giving a (relatively) poor prediction accuracy compared to more sophisticated models such as random forests or SVMs. Are there machine learning models commonly accepted as representing a good tradeoff between the two? Is there are any literature enumerating the characteristics of algorithms which allow them to be explainable?

EDIT: To make the notion of 'explainable' clearer, in interpreting a regression model, it is possible to make statements to the effect of 'an increase of x units of predictor i will lead to a y increase in the dependent variable'. By contrast, it is difficult to make a statement like that based on the output of a neural network. The first scenario is an example of a model being 'explainable'. This claim is made of linear regression in both Hastie, Tibshirani, Friedman's Elements of Statistical Learning (2008) and Kuhn, Johson's Applied Predictive Modelling (2013), although on review the actual word used 'interpretable'.

The tradeoff is that a regression model has far stricter assumptions than a neural network. For example, to give an accurate prediction, there must be little correlation between predictor variables, and the relationships between predictor variables and the dependent variables must be linear. If the data doesn't meet these (and other requirements), the predictions will not be as accurate. In contrast, there are models such as neural networks which require fewer assumptions (e.g. they may tolerate non-linearity far better), and achieve a better prediction accuracy, at the cost of losing 'explainability' as described above.

An example is given in the Compressive Strength Case Study in Kuhn, Johnson's Applied Predictive Modelling (2013), where the models the text had earlier labelled as most explainable - CART, linear regression - achieved the worst prediction accuracy on the data. A second set of data was also discussed in the text - and again neither C&RT nor linear regression were considered 'high performers' compared to, for example, boosted trees.


closed as primarily opinion-based by Tim, Nick Cox, Greenparker, John, Christoph Hanck May 21 '16 at 16:56

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ Your question is very subjective and opinion-based. What do you mean by "good explainability". Why do you consider regression to be "poor" (comparing to what? in what cases?). $\endgroup$ – Tim May 21 '16 at 10:34
  • $\begingroup$ I have added some additional comments that hopefully reduce the subjectivity - the question is poorly phrased if it leads to the idea that I 'consider regression to be poor'. $\endgroup$ – Robert de Graaf May 21 '16 at 12:04
  • $\begingroup$ This may be of your interest: stats.stackexchange.com/questions/6/… $\endgroup$ – Tim May 21 '16 at 13:12
  • 1
    $\begingroup$ To offer a dissenting voice to the close voters, here's why I vote to leave open: The question of whether there are models accepted as good tradeoffs seems to me a point of fact on current practice. Likewise to the question of whether related literature exists. (Put another way, whether e.g. LASSO provides good tradeoff between performance and interpretability is a matter of opinion; the commonly accepted view among practitioners is, I think, a question of data.) $\endgroup$ – Sean Easter May 21 '16 at 16:34
  • 1
    $\begingroup$ @RobertdeGraaf You may want to check here and follow the trail of citations forward and back. $\endgroup$ – Sean Easter May 25 '16 at 14:02