I'm working on a behavioral scorecard modelling exercise, and many of the decisions taken to date have been based on the experience of a consulting credit analyst (whose experience software-wise is SAS) as I am primarily in BI. So far I have:
- a linux pc with 32gb of ram and an i7 processor
- an observation window
- ~90 potential characteristics
- a binary outcome
In R, I have
- loaded the dataset (225k obs of 88 vars, 1 outcome)
- split the dataset up based on the recommendations/examples in the package caret i.e. predictors and outcomes split up (150k obs in training sample)
- removed any variables showing a high degree of correlation (caret::findCorrelation)
- cut all continuous variables into categorical intervals
- reduced the number of variables based on near zero variance, missing values, and low information value (IV) (150k obs of 48 vars)
- tried bestglm::bestglm, caret::train (with glm and glmnet), FWDselect::selection, FWDselect::qselection but eventually had to interrupt each of these due to not completing after 4 hours of 100% CPU usage
- used FactoMineR:MCA to perform a multiple correspondence analysis (on predictors only)
What I would like to do is have a selection of logistic regression models for say 4, 8, 12, and 16 variables that are the most predictive models at each point. I'm not sure if I'm going in the correct direction here with MCA as I've mainly been simply trying to find something that works in a timely fashion for reducing my variables further or going directly to variable selection steps.
I would appreciate any advice on how to do any of step 6 better, whether 7 makes sense and what step 8 should be.
PS Design decisions up to 6&7 can't be revised so please, no telling me off for them!