7
$\begingroup$

I'm running a RandomForest in R on a set of data with many variables. Using varImpPlot() I know how important is each variable to explain the target variable. However, I would also like to know whether a variable affects positively or negatively the target value.

For example, in predicting personal income, I may get that "age" explains a lot, but I would like to know whether "higher age -> higher income", or whether "lower age -> higher income".

Is there any way to know this with Random Forest?

$\endgroup$
2
5
$\begingroup$

As the answers at the linked question by @Simone show, it is entirely possible to quantify the partial effect of a predictor/independent variable on the target/dependent variable. This is a very useful way to derive understanding from a random forest (or machine learning model in general).

However, this is not exactly the same as what this question asks about i.e. quantify 'whether a variable affects positively or negatively the target value'. The reason is simple: random forests behave differently from linear models, which capture nonlinearities only when specified in the model formulation. In a simple linear model without quadratic terms - assuming no problems with the model fit - a positive slope estimate indicates that increases in a predictor variable lead to increases in the target variable. In contrast, random forests capture highly nonlinear partial effects without any prior specification.

Therefore, to use the example in the question, it's entirely possible for a random forest to capture the reality that income is low when young, high at middle age, and low again when old (i.e. post-retirement). In fact, random forests' structural flexibility means that they can capture far more complex nonlinear shapes than the simple one in this example. As a consequence of this flexibility, it would be unusual (though not impossible) for single predictors to have an effect that can accurately described as being just positive or negative. Plotting out the partial effects will give you a more complete and accurate impression.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.