Choosing Interpretable Models model vs choosing black box model and explain it with shap/lime

Question

I analyse a dateset of article. The articles are labeled as popular or not popular and off course each article has features like: article section, article writer and etc. I don't want to predict if a new article and unlabeled one will be popular or not. I want to explain which features and which values are associated with the unpopular articles. I Think about two options: use an interpretable Model (like logistic regression)or use black box model (like random forest) and explain it with lime or shap.

What option do you think is better?

Thanks

Answer 1

This is a good question, I would be interested in other answers.

Random forests are more interpretable than may first appear see Obtaining knowledge from a random forest. As an example and as mentioned in one of the answers from that question, the random forest you can get the importance of variables, say using the importance function of the R randomForest package.

Conversely, using a single logistic regression model may seem more interpretable than it actually is since unless there really are those exact regressors generating the output (unlikely) then probably a few slightly different models will be acceptable or plausible. Further, changing one regressor will often effect other regressors in the model if they are present, and how much to change a regressor (one unit, one percent?) may not even make sense for a given regressor, with a further complication being the variability of regressors is unlikely to be the same.

I would fit both a logistic regression and a random forest and look for similarities and differences in their results, augmenting this with any other domain knowledge or plots that help elucidate the data.

Answer 2

Explanation and prediction are not the same (see Galit Shmueli's webpage on her research on the topic, her paper published in Statistical Science (pdf), and Practical thoughts on explanatory vs. predictive modeling here on CV). You "don't want to predict if a new article and unlabeled one will be popular or not", which means you aren't really after prediction.

Instead, you "want to explain which features and which values are associated with the unpopular articles". That sounds like you want to be able to advise someone (say a potential author) how to write their article better so that it won't be unpopular. You are interested in causality. To determine that, you need to run experiments. You need to find some preliminary theoretical framework on the topic (there is presumably existing research, or you could use your intuitions). Then gather some authors and randomly instruct them to write articles with or without certain features (or in varying amounts). Then have readers rate their interest (as a proxy for popularity). With those data, you can fit simple logistic regression models and test if the features are significant.

There are also various methods for trying to infer causality from observational data, but it's a little dicier and the statistical techniques are more complicated. You can read some of our threads on the topic under the causality tag.