3
$\begingroup$

I'm wondering if there is any problem in using the predicted values (based on OOB observations) of a randomForest object as a predictor in the prediction of another variable. Something like this using R:

rf_interim <- randomForest(z ~ x1 + x2 + x3)
z_pred <- rf_interim$predicted
rf <- randomForest(y ~ x1 + x2 + x3 + z_pred)
rf$predicted

Ulitmately on new data, I want to predict z in a first random Forest and use that prediction to predict y in a second random Forest. Will it for example skew the error prediction or mess with any of the other outputs? Does this approach make sense when z is a strong predictor for y but not initially observable?

$\endgroup$
9
  • $\begingroup$ is there any motivation for this? $\endgroup$
    – utobi
    Commented Nov 3, 2022 at 13:54
  • 1
    $\begingroup$ So basically you have access to x1, x2, x3 on Monday, but you don't have access to z until Tuesday and you can't wait until Tuesday to predict y? $\endgroup$
    – Sycorax
    Commented Nov 3, 2022 at 13:57
  • $\begingroup$ @Sycorax Yes exactly. z and y are only observable later, not when I'm actually using the model to predict. $\endgroup$ Commented Nov 3, 2022 at 14:06
  • $\begingroup$ @utobi z is a strong predictor for y, which I ultimately want to predict. x1, x2 and x3 do well in predicting z not so much in predicting y. $\endgroup$ Commented Nov 3, 2022 at 14:08
  • 1
    $\begingroup$ (+1 fun question - and welcom to CV.SE). Yeah, that is probably not harmful but given what we want to do is $y=f^Y_{\text{t=2}}(x_1,x_2,x_3,x_4)$ but we ultimately have to use $f^Y_{\text{t=1}}(x_1,x_2,x_3,f^{x_4}_{\text{t=1}}(x_1,x_2,x_3))$, we should first use $y=f^{Y}_{\text{t=1}}(x_1,x_2,x_3)$ as a relevant baseline. Short of doing that first we somehow "hope" that the error in the prediction of $\hat{x}_4$ is not too detrimental. Similarly, we probably need to train our $f^Y_{\text{t=2}}(x_1,x_2,x_3,x_4)$ on purposely noisy $x_4$ otherwise we will face covariate shift when we validate. $\endgroup$
    – usεr11852
    Commented Nov 3, 2022 at 15:32

1 Answer 1

0
$\begingroup$

Okay, first observed issue with this approach: since leakage is probably an issue when using OOB's twice, we need to assess performance on a test set instead on OOB's. Something like this:

rf_interim <- randomForest(z ~ x1 + x2 + x3, data=train)
z_pred <- rf_interim$predicted
rf <- randomForest(y ~ x1 + x2 + x3 + z_pred, data=train)
predictions <- predict(rf, test)
mse <- sum((predictions - test$y)^2) / length(predictions)

So we don't use the rf$mse (uses OOB's) to assess performance but the MSE of the test set.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.