I am running
cv.glmnet from the
glmnet package in
R. I am running 10-fold cross validation 100 times on a dataset that has 25,000 observations and 150 variables. I am using the
parallel = TRUE option, as that greatly speeds up results.
- I notice that the minimum lambda value (
lambda.min) and the lambda value within 1 standard error of the minimum (
lambda.1se) are exactly the same in 99.9% of all 100 cross validation runs.
- Each cross validation run leads to the calculation of exactly 25 lambdas - no more and no less (recall that the default is that the number of lambdas depends on whether the deviance changes sufficiently from one run to the next). I have left the default settings as they are, which allows for the calculation of 100 lambdas unless the deviance stops changing significantly.
- The cross validation error varies minimally (average of 0.002).
- If I run another few iterations of 100 10-fold cross validations on a different split into training and test populations, I get a slightly different minimum lambda and lambda within 1 standard error of the minimum from the other iterations. The above points hold, however, for the given iteration (i.e. same lambda, same exact number of lambdas calculated per cross validation run, minimal change in cross validation error).
Why are all my cross validation results within a given iteration so similar (perhaps because all folds in the cross validation are always very similar)?