# Why Does a Variable with Weak Correlation to Outcome Variable Emerge as Most Important Factor in Random Forest?

I'm puzzled about why a dependent variable with the weakest correlation to the outcome variable emerges as the most important factor when I run my random Forest on the same dataset. It beats out factors with much stronger correlations with the outcome variable.

Now I'm not sure whether to trust my random Forest. All the predictor variables were numeric and the outcome variable was a factor variable with two levels.

Do you have insight about 1) why this happens and 2) what I should do next to assess which variables are in fact most predictive?

The RF code I used was:

rf_mod <- randomForest(Stable ~ ADF + EstAnnualVal + PR_Score + stdv_demand_intervals,
data=var1, ntree=501, importance=TRUE)

• I'm guessing you have interaction effects between one or more variables, such that there might be almost no correlation of a to b directly, but there might be between a and b at each level of c May 29 '16 at 22:41
• You haven't been very specific about how you've calculated correlation and importance. In any case there's a general issue: are you interested in how predictive each variable is on its own, or in how much predictiveness it adds on top of the other variables? Those are quite different questions. May 30 '16 at 14:44

Correlation of the variable with the dependent one has the same assumption as Naive Bayes - independence of the variables. Thus, the only thing you are able to test this way is "in the absence of all other features, how well would I describe my dependent variable, if I fit a monotonic model to this one, selected feature". These are two, extremely strong, assumptions. Consequently you completely omit any, even a bit complex, relation. For example if you have a numeric variable, such that every time it is odd, your true label is 1, and every time it is even it is 0 - your correlation test will say "there is nothing interesting in this variable", while you have full knowledge about dependent variable, as it is y = x % 2. Furthermore, even if the relation is monotonic and simple, but involves multiple factors - you will miss it too!