In trainControl of Caret, how to keep a specific proportion of samples for cross validation?

Question

Sample Data:

trainX <- mtcars %>% select(-c("vs", "am", "gear", "carb"))
trainY <- as.factor(mtcars$vs)
levels(trainY) <- list(yes=1, no=0)

The expected proportion of data

proportions(table(trainY))

trainY
   yes     no 
0.4375 0.5625 

Model fit

tcontrol_1  <- trainControl(method="cv", number=10, classProbs=TRUE, summaryFunction=twoClassSummary)
model_fit   <- train(x=trainX, y=trainY, method = "glmnet", family = "binomial", standardize=TRUE, trControl = tcontrol_1, metric = "Sens")

Now, in the 10 fold CV, for each fold, how can I keep that proportion of samples?

Does trainControl do it by default?

Thanks in advance.

Answer 1

I doubt you can; you certainly cannot precisely since you have $0.4375 = \frac{7}{16}$ and none of the folds have a length which is a multiple of $16$.

And I am not sure why you would want to: each fold is going to train on most of the data (about $90\%$ in your example) so will reflect something close to the overall proportion. For example

library(caret)

trainX <- mtcars %>% select(-c("vs", "am", "gear", "carb"))
trainY <- as.factor(mtcars$vs)
levels(trainY) <- list(yes=1, no=0)

set.seed(2021)
tcontrol_1  <- trainControl(method="cv", number=10, classProbs=TRUE, 
                summaryFunction=twoClassSummary)
model_fit   <- train(x=trainX, y=trainY, method = "glmnet", 
                 family = "binomial", standardize=TRUE, 
                   trControl = tcontrol_1, metric = "Sens")

prop <- function(x, Y=trainY){ proportions(table(Y[x])) }
prop(1:length(trainY))
#    yes     no 
# 0.4375 0.5625 
sapply(model_fit$control$index, prop)
#        Fold01    Fold02    Fold03    Fold04    Fold05    Fold06    Fold07
# yes 0.4285714 0.4482759 0.4137931 0.4482759 0.4285714 0.4137931 0.4482759
# no  0.5714286 0.5517241 0.5862069 0.5517241 0.5714286 0.5862069 0.5517241
#        Fold08    Fold09    Fold10
# yes 0.4482759 0.4482759 0.4482759
# no  0.5517241 0.5517241 0.5517241

so they are close but not exact for the training folds. For the validation folds, they are even less close (with about three observations per fold, what would you expect?) but there does seem to be a fix to avoid total imbalance.

sapply(model_fit$control$indexOut, prop)
#     Resample01 Resample02 Resample03 Resample04 Resample05 Resample06
# yes        0.5  0.3333333  0.6666667  0.3333333        0.5  0.6666667
# no         0.5  0.6666667  0.3333333  0.6666667        0.5  0.3333333
#     Resample07 Resample08 Resample09 Resample10
# yes  0.3333333  0.3333333  0.3333333  0.3333333
# no   0.6666667  0.6666667  0.6666667  0.6666667