# How to implement a hold-out validation in R

Let's say I'm using the Sonar data and I'd like to make a hold-out validation in R. I partitioned the data using the createFolds from caret package as folds <- createFolds(mydata\$Class, k=5).

I would like then to use exactly the fold mydata[i] as test data and train a classifier using mydata[-i] as train data.

My first thought was to use the train function, but I couldn't find any support for hold-out validation. I mean, I don't want to partition my data and then test all the combinations for estimating the mean accuracy, I want to train it only once. Am I missing something here?

Also, I'd like to be able to use exactly the pre-defined folds as parameter, instead of letting the function partition the data. Does anyone have any thoughts?

It is hard to tell exactly what you are interested in. If it is a single hold-out, you can use

trainControl(method = "LGOCV", p = .8, number = 1)


and 80% will be used for training.

There is also method = "none" that will just fit the model for a single tuning parameter value (using the entire training set).

Also, if you want to use your own hold-out set(s), see the index argument of trainControl.

Max

• It worked!! Thanks a lot, @topepo, and great job with the package! I noted, however, that if i pass an index with the method="knn", it gives an error. Is that expected? – gcolucci Apr 10 '14 at 23:31