I am reading a chapter about random forest in a textbook. After the section about the predictor importance, the author introduces "Response Functions" as follow:

"Predictor importance is only part of the story. In addition to knowing the importance of each predictor, it can be very useful to have a description of how each predictor is related to the response. The set of response functions needs to be described. For tree-based approaches such as CART, one proceeds as follows.

  1. Grow a forest.
  2. Suppose x1 is the initial predictor of interest, and it has v distinct values in the training data. Construct v data sets as follows.

    a) For each of the v values of x1, make up a new dataset where x1 only takes on that value, leaving all other variables untouched.

    b) For each of the v datasets, predict the response using random forests.There will be a single value averaged over all observations.

    c) Average each of these predictions over the trees.

    d) Plot the average prediction for each value for each of the v datasets against the v values of x1

  3. Go back to Step 2 and repeat for each predictor."

I use random forest occasionally, and I understand the basic procedure of the model. But the step 2 b) and 2 c) are confusing to me. If we have N observations and x1 has p unique values, can anyone help to describe how the steps should be done?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.