2
$\begingroup$

Hi Dear Colleagues,

I wonder how to correctly setup a leave-one-subject-out cross validation (LOSO) for train() function in caret.

Here is my example code:

dat <- as.data.frame(cbind(rnorm(1:500,1),rnorm(1:500,10),rnorm(1:500,5),
          rnorm(1:500,100),c(rep('1',100),rep('2',100),rep('3',100),rep('4',100),
                                rep('5',100)),rep(c(rep('X0',50),rep('X1',50)),5)))

colnames(dat) <- c('var1', 'var2', 'var3', 'var4', 'subject','class')

What would you then specify in trainControl() instead of cv (LOGCV?)? How would you modify the following code?

svmFit <- train(dat,y,
                method = "svmRadial",
                preProc = c("center", "scale"),
                tuneGrid = MySVMTuneGrid,
                trControl = trainControl(method = "cv", number = 10, classProbs =  TRUE))

Thank you very much for your time!

$\endgroup$

1 Answer 1

3
$\begingroup$

If there is one subject per row, then method = LOOCV would do it. You will have to setup your own resampling indicators and supply them via index. At that point, the value of method does't matter.

You could do something like:

subs <- unique(dat$subject)
model_these <- vector(mode = "list", length = length(subs))
for(i in seq_along(subs)) 
   model_these[[i]] <- which(dat$subject != subs[i])
names(model_these) <- paste0("Subject", subs)
svmFit <- train(class ~ var1 + var2 + var3 + var4,
                data = dat,
                method = "svmRadial",
                preProc = c("center", "scale"),
                tuneGrid = MySVMTuneGrid,
                trControl = trainControl(method = "cv", 
                                         index = model_these, 
                                         classProbs =  TRUE))

(note that your test data set converts var-var4 to character, so I didn't test this.)

Max

$\endgroup$
1
  • $\begingroup$ i have a problem when i run this code. Error: index should be lists of integers.That error appears. $\endgroup$ Commented May 10, 2021 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.