section 14.4 in pattern recognition and machine learning (free) gives this figure

enter image description here

and says

Within each region, there is a separate model to predict the target variable. For instance, in regression we might simply predict a constant over each region, or in classification we might assign each region to a specific class. A key property of tree-based models, which makes them popular in fields such as medical diagnosis, for example, is that they are readily interpretable by humans because they correspond to a sequence of binary decisions applied to the individual input variables. For instance, to predict a patient’s disease, we might first ask “is their temperature greater than some threshold?”. If the answer is yes, then we might next ask “is their blood pressure less than some threshold?”. Each leaf of the tree is then associated with a specific diagnosis.

In order to learn such a model from a training set, we have to determine the structure of the tree, including which input variable is chosen at each node to form the split criterion as well as the value of the threshold parameter $θ_i$ for the split. We also have to determine the values of the predictive variable within each region.

consider region A, in which instances have $x_1 \leq θ_1$ and $x_2 \leq θ_2$, within this specific region, what the predictive variable is?


It refers to the prediction you assign to each region when constructing the tree during training. In a classification task it could be the majority label of all examples in region A. In a regression task it could be the average of the true $y$ values of all examples in region A.

Once you're done with training and applying the tree, all examples being allocated to A will be predicted as that predictive value, i.e. it determines $\hat{y}$ for all examples in region A.

Here is a concrete example: if in a classification task during training region A contains 7 examples classified as "dog" and 3 example classified as "cat", then you would probably choose "dog" as predictive value of region A. Following that, when applying the model to test data or real data all examples being allocated to region A would be predicted as "dog".

  • $\begingroup$ Thanks for your answer. So, prediction = predictive variable = predicted value = the output of a trained decision tree regarding to a specific region such as region A, right? $\endgroup$
    – JJJohn
    Dec 15 '19 at 0:15
  • $\begingroup$ Yes. It is a bit confusing how they use the phrase in the book since one might think "predictive variable" refers to your predictors aka independent variables. But they use it differently. @JJJohn $\endgroup$
    – Sammy
    Dec 15 '19 at 8:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.