2
$\begingroup$

This is a more theoretical question, perhaps less suiting to the required format of a question on cross validated. Still I hope someone can help or advise me.

I'm using r.

I have a data frame with roughly 180k observations and 120 variables. When running through a multiple regression model there ends up being ~1000 variables including dummy variables.

My baseline kitchen sink model is the target variable as a function of all ~999 variables. This takes r about 10 minutes to compute on a single 0.8 train and 0.2 test data on my 6gb Mac, 2.2 processor.

On my baseline kitchen sink model I also did kfolds cross validation on training with 10 folds. Roughly this took an hour to compute.

I'm looking at my notes from school on feature selection. I would like to try stepwise regression, forwards and backwards to see what it comes with for my performance measure Mean Absolute Error.

I would also, for each iteration within stepwise, train using cross validation.

Is this possible? If running the baseline model 10 times with a single cross validation takes 1hour then I'm about to ask r to do the same 999 times with a stepwise instruction.

Is what I'm trying to do "normal"? Is my understanding and approach "sane"? Given the size of my data and that I'm on a 6gb Mac, what would be a logical approach for feature selection and training?

If I should provide any more info here such as str(df) let me know.

$\endgroup$
1
  • $\begingroup$ For a speed boost look up the lmfit function in R, and use it instead directly with matrixes instead of a formula based model $\endgroup$
    – Repmat
    Oct 14, 2016 at 11:09

1 Answer 1

1
$\begingroup$

Instead of using stepwise regression, which is a method very often frowned upon, I would use LASSO/Elastic Net, gamboostLSS or Bayesian Model Averaging.

If you would like to gauge the models predictive performance then a kfold cv is a good way to go about doing so.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.