repeated cross-validation shows similar results for different k-folds

Question

I need to see whether different k-fold cross-validations with repeats will improve my classification accuracy.

For that, I have tested with repeatedCV (k-fold=50, repeat=10, k-fold=100, repeat=10, and k-fold=1000, repeat=10)

But, accuracy values in the confusion matrix are same for the above different repeated cross-validations.

Please see my code below. I am using caret package for training the data.

I am thinking whether I am getting the same accuracy results because my my model is stable/robust or due to some other reason.

Any help is appreciated.

Thanks in advance.

standardized.X <- LossT[,-c(1:10)]
set.seed(50)
ind <- createDataPartition(LossT$LRPPcat, p=0.85, list = FALSE)
train.X <- standardized.X[ind,]
test.X <- standardized.X[-ind,]
train.Y=LossT$LRPPcat[ind]
test.Y=LossT$LRPPcat[-ind]

set.seed(123)
seeds <- vector(mode = "list", length = 501)
for(i in 1:501) seeds[[i]] <- sample.int(1000, 20)
## For the last model:
seeds[[501]] <- sample.int(1000, 1)

# Train the model
ctrl <- trainControl(method = "repeatedcv",number = 50, repeats = 10, preProcOptions = list(thresh = 0.85, k = 5),seeds = seeds, selectionFunction = "best", savePredictions = TRUE, allowParallel = TRUE) 
modelknn <- train(train.X,train.Y, method = "knn",preProcess = c("pca"), trControl = ctrl,metric = "Accuracy",tuneLength = 20)

fitted.results <- predict(modelknn,test.X) 
cm <- confusionMatrix(data=fitted.results, reference=test.Y) 

Answer 1

But, accuracy values in the confusion matrix are same for the above different repeated cross-validations.

This is how it should ideally be. The actual choice of the number of folds does not influence the results much, see also Choice of K in K-fold cross-validation

I am thinking whether I am getting the same accuracy results because my my model is stable/robust or due to some other reason.

The aggregated confusion matrix (summing up the results of all surrogate models) is not the best place to characterize stability [of the predictions wrt. small changes in the training set]. You can get that more directly by comparing predictions across the repetitions.

The "other reason" may be that you have many cases (as you do k = 1000), so the random (relative) differences you observe in the final result are typically smaller than those observed with small data sets.