# Machine learning/random forests with noisy response data

Machine learning techniques like random forests seem to assume that the responses in the training set are known perfectly. Specifically for regression applications, it seems one needs to account for the measurement error (which may be well characterized) in the responses (or the predictors, for that matter). In classical regression, you can add inverse variance weights to address this, but I'm not sure how to translate that to machine learning techniques.

I can imagine doing Monte Carlo, say 1000 random forests each with 1000 trees, in which I sample the response variables from their (known) distributions, and then aggregating the results. But that seems rather brute force.

Are there machine learning techniques in which you can explicitly account for uncertainty in your variables? Ideally, I'd also like the machine learning prediction to include these uncertainties (i.e., predict a confidence interval).

• The Monte Carlo approach could be made more efficient if sampling the response variable from its known distribution were done for each tree, i.e. when bootstrapping the data -- instead of $F>1$ forests with $T>1$ trees, you would only need 1 forest with $T$ trees. – Sycorax says Reinstate Monica Mar 26 '18 at 15:53

In an unweighted RF, $n$ random subsamples of points are selected, and regression trees (jointly comprising the forest) are independently fit to each of them. Weights are incorporated in the random forest by altering the probabilities with which points are selected in each random subsample. A point with a higher weight will be present in a greater proportion of trees, and consequently have a larger influence on the random forest.