1
$\begingroup$

I have a regression problem where I’m attempting to train a data set with 70 predictors, but only 35 observations with glmnet in the caret package. I’m trying to determine the best resampling method. It seems like 35 observations is too few to use K-Fold Cross Validation. Would bootstrapping be a better method? What we would be the ideal number of iterations for either method? My biggest concern is overfitting. Is there some general guideline to handle these type of data sets?

Are there better models for these type data of sets? Random Forests or Boosting come to mind.

$\endgroup$
1
$\begingroup$

This is a tiny sample size and unlikely to be useful unless you have a very high signal to noise ratio. I would rank the predictors in terms of their correlation with the outcome, and use the top few (say top 1-2). You could bootstrap to get an idea of the variability, but it's likely to be high. Then I'd spend more time trying to find more data or another independent dataset to validate on. No point in using sophisticated approaches because they will just overfit.

Edit: leave-one-out validation may also be somewhat useful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.