I have a dataset with 600 rows and 4000 columns for which I am trying to do held-out cross-validation with 10 and 100 iterations.
At first, the dataset is split into 80%:20% training and “held-out” test sets. Five-fold cross-validation was done. This algorithm was then used for prediction on the “held-out” 20% data in the test set. This process was repeated N = 10/100 times.
For held-out, with an increase in the number of iterations, the AUC curve is going down. For 10, iterations I am getting a mean AUC of 0.80, whereas for 100 iterations I am getting a mean AUC of 0.67.
What could be the possible reason for this decrease in AUC values?
Is it due to the parameter tunning?
train_control <- trainControl(method="cv", number=10) svm.model <- svm(Class ~ ., data = training,metric="ROC", ranges=list(cost=10^(-1:2), gamma=c(.5,1,2)), type="eps",trControl=train_control,kernel="radial",na.action=na.omit,probability = TRUE)
Is it the right notation to get the list of cost and gamma values?