I am reading a chapter about random forest in a textbook. After the section about the predictor importance, the author introduces "Response Functions" as follow:
"Predictor importance is only part of the story. In addition to knowing the importance of each predictor, it can be very useful to have a description of how each predictor is related to the response. The set of response functions needs to be described. For tree-based approaches such as CART, one proceeds as follows.
- Grow a forest.
Suppose x1 is the initial predictor of interest, and it has v distinct values in the training data. Construct v data sets as follows.
a) For each of the v values of x1, make up a new dataset where x1 only takes on that value, leaving all other variables untouched.
b) For each of the v datasets, predict the response using random forests.There will be a single value averaged over all observations.
c) Average each of these predictions over the trees.
d) Plot the average prediction for each value for each of the v datasets against the v values of x1
- Go back to Step 2 and repeat for each predictor."
I use random forest occasionally, and I understand the basic procedure of the model. But the step 2 b) and 2 c) are confusing to me. If we have N observations and x1 has p unique values, can anyone help to describe how the steps should be done?