For those of you familiar with Exploratory Factor Analysis (EFA) and Random Forest (RF), I have recently had an idea of combining these two methods to reduce the number of potential predictor variables for use in a parsimonious binary logistic regression model. For the purposes of this post, assume large n (200k or more) and 1000 potential predictor variables.
To employ this idea, the first step would be to perform an EFA with all potential predictor variables using proc varclus
. Additionally, using randomForest
to rank all potential predictor variables by IncNodePurity
(Gini Index).
After these two methods are independently used, I propose retaining the variable with the largest IncNodePurity
(from RF) within each factor (from EFA).
Does anyone have any thoughts/concerns with this methodology (or lack thereof) for feature selection? I am aware that this "picking and choosing" of methods may be complete garbage, but I had this random thought and wanted to share. Thanks!