Standard deviation in regression trees

Question

In a regression tree, it is often assumed that each leaf is a Gaussian distribution $\mathcal{N}(\mu_i, \sigma)$, where $i$ is the index the leaf. Is $\sigma$ calculated as the standard deviation within a leaf or the standard deviation for the dataset?

If a tree is grown such that each leaf contains one instance (as is the case for bagged trees) then the within leaf calculation seems ill-posed. However, using the whole dataset seems like it could induce a lot of bias.

Is there something I am missing here?

Answer 1

The way I understand it, the standard deviation $\sigma$ is calculated within a leaf. This is because, at each split, we are trying to minimise the sum total of the population variance, where the population is the collection of target values at each leaf. It is not ill-posed (in the case where a leaf contains one instance) as we are not trying to calculate a sample variance as an estimate for the population variance. For the case where a leaf contains one target value $\{y\}$, the mean is $y$ and the variance is zero.