I have Data X1, X2, and y. X1 has the same variables as X2, + some extra variables that X2 does not have.

I want to use the data X2 to predict binary variable y. I suspect the extra variables In X1, that are not in X2 to have a lot of predictive value

Consider the following two procedures:

Procedure 1: Use X2 to predict Y

Procedure 2: First ake a model that predicts the second set of variables in X1 with the first set of variables in X1. Use this model to predict the extra variables for data X2. Then use X2+the fitted values for the extra variables to predict Y

Question: Can Procedure 2 lead to a better prediction than Procedure 1? Or are the two procedures essentially the same?


1 Answer 1


Did a small empirical test with simulated data.

Seems like procedure 2 can improve performance when using Random Forests.

After thinking about it a bit more i think you dont really add more information into the model by using procedure 2 but it can help to present the information in a way that makes it easier for the algorithm to make a correct classification. Just like PCA can improve performance sometimes although it doesnt add new info.

Seems like for linear models procedure 2 wont add anything or even make it impossible for the method to run, e.g iirc regression doesnt work when one of the features is a linear combination of the others.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.