How can I see the influence of the features in my model (e.g. what is the impact of a specific feature on the target variable)?

Saw a great article ( "Making the XGBoost interpretable", but that package is not working.

So, If I build a model (lets say I use Random Forest, Boosting tree ...). How can I see the impact of each variable (I'm using R most of the time).

I only know the VarImp() function, which give the importance of a feature (in a scale 1 - 100, but not if the impact is positive or negative:

  • Could you elaborate on what you mean by "impact" or "importance"? I can immediately think of many distinct, well-known concepts that these might refer to (sensitivity, effect size, significance, influence, leverage). – whuber Feb 1 at 15:55
  • 1
    @whuber I like the way you make comments. many of my questions got answered by myself when I try to clarify the question from from your comments – hxd1011 Feb 1 at 15:58
  • @Whuber, sure: let's say I made a model with as target: sales = Yes or No. To predict the sales, I use the features "Campaign" and "Price". What I want to know, is what the influence of "Campaign" or the "Price" is on Sales (so do I have to invest in more Campaign, or is the price important. And if, how important are those features. For now I can predict stuff, but I have no idea which features are driving my target. Does this provide a bit more clarity? – R overflow Feb 1 at 16:02
  • 1
    Whubers point is basically that words like "influence" and "importance" are not defined. So by "influence" do you mean: the effect of the predictions for sales as a vary campaign and price? – Matthew Drury Feb 1 at 16:07
  • 1
    I thought that you meant something else. But then we are on the same page :-) @Matthew Drury – R overflow Feb 1 at 17:02

How best to describe a model's predictions in terms of the input variables depends on the model. In the case of linear regression, for example, this question is directly answered by the coefficients. Neural networks, by contrast, tend to be black boxes with no easy way to see how the input relates to the output. A model-independent way to approach this question is the experimental approach of asking the model to make predictions for fake data you've made in which you vary a single input variable of interest.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.