# Using $predicted values of a randomForest object as predictor in training for another randomForest I'm wondering if there is any problem in using the predicted values (based on OOB observations) of a randomForest object as a predictor in the prediction of another variable. Something like this using R: rf_interim <- randomForest(z ~ x1 + x2 + x3) z_pred <- rf_interim$predicted
rf <- randomForest(y ~ x1 + x2 + x3 + z_pred)
rf$predicted  Ulitmately on new data, I want to predict z in a first random Forest and use that prediction to predict y in a second random Forest. Will it for example skew the error prediction or mess with any of the other outputs? Does this approach make sense when z is a strong predictor for y but not initially observable? • is there any motivation for this? Commented Nov 3, 2022 at 13:54 • So basically you have access to x1, x2, x3 on Monday, but you don't have access to z until Tuesday and you can't wait until Tuesday to predict y? – Sycorax Commented Nov 3, 2022 at 13:57 • @Sycorax Yes exactly. z and y are only observable later, not when I'm actually using the model to predict. Commented Nov 3, 2022 at 14:06 • @utobi z is a strong predictor for y, which I ultimately want to predict. x1, x2 and x3 do well in predicting z not so much in predicting y. Commented Nov 3, 2022 at 14:08 • (+1 fun question - and welcom to CV.SE). Yeah, that is probably not harmful but given what we want to do is$y=f^Y_{\text{t=2}}(x_1,x_2,x_3,x_4)$but we ultimately have to use$f^Y_{\text{t=1}}(x_1,x_2,x_3,f^{x_4}_{\text{t=1}}(x_1,x_2,x_3))$, we should first use$y=f^{Y}_{\text{t=1}}(x_1,x_2,x_3)$as a relevant baseline. Short of doing that first we somehow "hope" that the error in the prediction of$\hat{x}_4$is not too detrimental. Similarly, we probably need to train our$f^Y_{\text{t=2}}(x_1,x_2,x_3,x_4)$on purposely noisy$x_4$otherwise we will face covariate shift when we validate. Commented Nov 3, 2022 at 15:32 ## 1 Answer Okay, first observed issue with this approach: since leakage is probably an issue when using OOB's twice, we need to assess performance on a test set instead on OOB's. Something like this: rf_interim <- randomForest(z ~ x1 + x2 + x3, data=train) z_pred <- rf_interim$predicted
rf <- randomForest(y ~ x1 + x2 + x3 + z_pred, data=train)
predictions <- predict(rf, test)
mse <- sum((predictions - test$y)^2) / length(predictions)  So we don't use the rf$mse (uses OOB's) to assess performance but the MSE of the test set.