Machine learning techniques like random forests seem to assume that the responses in the training set are known perfectly. Specifically for regression applications, it seems one needs to account for the measurement error (which may be well characterized) in the responses (or the predictors, for that matter). In classical regression, you can add inverse variance weights to address this, but I'm not sure how to translate that to machine learning techniques.

I can imagine doing Monte Carlo, say 1000 random forests each with 1000 trees, in which I sample the response variables from their (known) distributions, and then aggregating the results. But that seems rather brute force.

Are there machine learning techniques in which you can explicitly account for uncertainty in your variables? Ideally, I'd also like the machine learning prediction to include these uncertainties (i.e., predict a confidence interval).

  • 1
    $\begingroup$ The Monte Carlo approach could be made more efficient if sampling the response variable from its known distribution were done for each tree, i.e. when bootstrapping the data -- instead of $F>1$ forests with $T>1$ trees, you would only need 1 forest with $T$ trees. $\endgroup$ – Sycorax Mar 26 '18 at 15:53

You mention weighting points in linear models as a method of incorporating uncertainty. This is possible to do in random forests as well.

In an unweighted RF, $n$ random subsamples of points are selected, and regression trees (jointly comprising the forest) are independently fit to each of them. Weights are incorporated in the random forest by altering the probabilities with which points are selected in each random subsample. A point with a higher weight will be present in a greater proportion of trees, and consequently have a larger influence on the random forest.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.