I have a regression problem where I’m attempting to train a data set with 70 predictors, but only 35 observations with glmnet in the caret package. I’m trying to determine the best resampling method. It seems like 35 observations is too few to use K-Fold Cross Validation. Would bootstrapping be a better method? What we would be the ideal number of iterations for either method? My biggest concern is overfitting. Is there some general guideline to handle these type of data sets?
Are there better models for these type data of sets? Random Forests or Boosting come to mind.