For those of you familiar with Exploratory Factor Analysis (EFA) and Random Forest (RF), I have recently had an idea of combining these two methods to reduce the number of potential predictor variables for use in a parsimonious binary logistic regression model. For the purposes of this post, assume large n (200k or more) and 1000 potential predictor variables.
To employ this idea, the first step would be to perform an EFA with all potential predictor variables using
proc varclus. Additionally, using
randomForest to rank all potential predictor variables by
IncNodePurity (Gini Index).
After these two methods are independently used, I propose retaining the variable with the largest
IncNodePurity (from RF) within each factor (from EFA).
Does anyone have any thoughts/concerns with this methodology (or lack thereof) for feature selection? I am aware that this "picking and choosing" of methods may be complete garbage, but I had this random thought and wanted to share. Thanks!