Direction of the z-scores in randomForest I wonder if someone could help me with the interpretation of the "z-scores" (%IncMSE) in the random forest approach. I am using a randomForest library in R. Specifically, I am not sure whether the sign of the score (-/+) can be possibly interpreted as a direction of the effect of a predictor on the outcome variable?
 A: First of all, those importance scores are not Z-scores (assuming you talk about the thing that comes out from importance(model,scale=TRUE)) in pivot sense; they are a mean of error increase over trees divided by its standard deviation, but the distribution is not normal so you can't say that the value >3 is significant or anything like this.
The sign does not mean effect direction; negative importance simply means that disturbing certain attribute improves the model. This is somewhat a counter-intuitive effect and usually happens if the attribute is a pure noise or when the whole model is weak or overfitted.
A: I think what you are looking for is measuring the partial dependance, from the vanilla random forest r implementation manual:

Description Partial dependence plot gives a graphical depiction of the
  marginal effect of a variable on the class probability
  (classification) or response (regression).

You can also find a little more formal description there. This can be a little tricky to use correctly so read about it a bit. This is an example provided there on the iris data set:
data(iris)
set.seed(543)
iris.rf <- randomForest(Species~., iris)
partialPlot(iris.rf, iris, Petal.Width, "versicolor")


