I read RFCDE: Random Forests for Conditional Density Estimation. Just like it sounds, these folks trained random forests for making conditional density estimates. At inference time, density estimates are calculated by creating a KDE from the weighted collection of training records from each leaf node that the new data fell into.

During training, it acts just like other random forest except it has a special splitting criteria:

$$L(f, \hat{f}) = \int\int(f(z|x)-\hat{f}(z|x))^2dz dP(x)$$ Where $z$ is the output variable and $x$ is the input variable.

In general, I'm looking for an intuitive explanation of this loss function. More specifically: why is there a double integral here? To me, it seems like a single integral of this expression over $x$ would make some sense as a way to "sum" up the differences over the full domain of the two functions. Further, I'm not sure what function $P$ represents. Any smartypants out there?

  • 2
    $\begingroup$ There are two variables being integrated over ($x$ and $z$), hence the two integrals. $P$ is probably a probability measure; the notation $d P(x)$ suggests a measure-theoretic integral $\endgroup$ – Artem Mavrin Mar 18 at 21:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.