single node tree explanation

Question

1. solution: you have a single node tree and you have this explanation. "Because of the inability to split the dataset on any variable, the average of the entire data set is applied as the estimate for all predictions."

For a single node tree, I know that you cannot make a split. You have a training set. let's say you have 6 observations. Then, from the quote, do you take average of the 6 observations and average all the predictions? can you please tell me if this is what the solution is saying?

2. If you have multiple terminal nodes, then you can make a split. Then, can you please advise me how the solution may say ?

Answer 1

A single node regression tree is sort of like a regression with only the intercept. The best you can do is predict that every observation will have the mean of the DV.

When you have more than one node, you can assign the mean for the observations in that node to every observation in that node.

(I'm saying "mean" because your question implies that that is the metric you are using).

You mention a tree with 6 observations. Maybe you were just using that as an example, but trees don't work well with so few observations.

Answer 2

can you please tell me if this is what the solution is saying?

Yes, in the case of a single node decision tree regressor, the prediction will always be the chosen metric (in this case the average) applied to the set of the target values in the observations used to build the tree, regardless of what the input is.

If you have multiple terminal nodes, then you can make a split. Then, can you please advise me how the solution may say ?

The split is usually done at the training phase. When you have already built a decision tree using a number of labeled observations, you go from one (parent) node to another (child) node by performing a test that was associated with the parent node at training time. In the case of a binary test (e.g. $X < \mu$), the parent node will have two child nodes, each corresponding to one of the two results of the test. When you have multiple terminal nodes, you would go through the process of "passing" your input from one node to the other using the previously mentioned strategy. Doing this, your input will eventually end up at one of the terminal nodes, and then, just like in the case of a single node decision tree, your prediction will be the mean of the target values of the observations associated with that node. This mean is also computed at the training phase, just like the tests associated with each node.