20
$\begingroup$

I am using the library caret in R to test various modelling procedures.

The trainControl object allows one to specify a re-sampling method. The methods are described in the documentation section 2.3 and include: boot, boot632, cv, LOOCV, LGOCV, repeatedcv and oob. Although some of these are easy to infer, not all of these methods are clearly defined.

What are the procedures corresponding to these resampling methods?

$\endgroup$
  • $\begingroup$ documentation link is broken. Use this instead. $\endgroup$ – vikas Dec 4 '16 at 2:46
20
$\begingroup$

Ok, here is my try:

  • boot - bootstrap
  • boot632 -- 0.632 bootstrap
  • cv -- cross-validation, probably this refers to K-fold cross-validation.
  • LOOCV -- leave-one-out cross validation, also known as jacknife.
  • LGOCV -- leave-group-out cross validation, variant of LOOCV for hierarchical data.
  • repeatedcv -- is probably repeated random sub-sampling validation, i.e division to train and test data is done in random way.
  • oob -- refers to out-of-bag estimation proposed by Breiman, which further is related to bootstrap aggregating. (The file in the link is not a ps file, but a ps.Z file, rename it and then try opening.)
$\endgroup$
  • 1
    $\begingroup$ I believe that LGOCV is random splitting between a training set and validation set, repeated n times. So, instead of the ordinary case of splitting data between train and hold-out (build model on train and validate on hold out) once, this process is repeated many times. $\endgroup$ – B_Miner Nov 11 '11 at 19:30
  • 3
    $\begingroup$ I also believe that repeatedCV is k-fold cross validation, done multiple times. $\endgroup$ – B_Miner Nov 11 '11 at 19:31
  • $\begingroup$ Hard to believe this isn't documented somewhere. $\endgroup$ – andrew Oct 16 '16 at 16:49
4
$\begingroup$

The repeatedcv is repeated 10–fold cross–validation for sure, according to Max Kuhn's presentation. The default resampling scheme is the bootstrap.

A good file that you can look about resampling methods is Predictive Modeling with R and the caret Package (pdf). Max presented this in "useR! 2013".

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.