i am learning data mining through book . During classification chapters about Neural Networks the authors have below code. I have below questions:

## pre2008 <- 1:nrow(training) ## training is a dataset that has training data
ctrl <- trainControl(method = "LGOCV",
                     summaryFunction = twoClassSummary,
                     classProbs = TRUE,
                     index = list(TrainSet = pre2008),
                     savePredictions = TRUE)
nnetGrid <- expand.grid(.size = 1:10, .decay = c(0, .1, 1, 2))
maxSize <- max(nnetGrid$.size)
nnetFit <- train(x = training[,reducedSet], 
                 y = training$Class,
                 method = "nnet",
                 metric = "ROC",
                 preProc = c("center", "scale"),
                 tuneGrid = nnetGrid,
                 trace = FALSE,
                 maxit = 2000,
                 MaxNWts = 1*(maxSize * (length(reducedSet) + 1) + maxSize + 1),
                 trControl = ctrl)

LGOCV - when do we use it? I read the post, but still not clear. the post says that it is a variant of LOOCV for hierarchical data. but my Y variable is not hierarchical :(

twoClassSummary - can it be used only when we have two classes? can i used it for say Iris data?

LGOCV is also known as Monte-Carlo Cross Validation. More details are available here.

  • $\begingroup$ A quick Google establishes "leave-group-out cross validation" as the answer to your first question. Other questions are all focused on software/programming and arguably off-topic here. $\endgroup$
    – Nick Cox
    Aug 27, 2013 at 17:30
  • $\begingroup$ the post that i have mentioned in my email says that LGOCV means "leave-group-out cross validation" , but i would like to know more details of it. for example: when we should use it and does it need Y to be hierarchical .. $\endgroup$ Aug 27, 2013 at 17:56
  • 1
    $\begingroup$ OK, but you said "what does it stand for?" and I answered that. Perhaps you should make your question clearer. $\endgroup$
    – Nick Cox
    Aug 27, 2013 at 17:59

1 Answer 1


From the book: "Repeated training/test splits is also known as 'leave-group-out cross- validation' or 'Monte Carlo cross-validation.'". It is illustrated in Figure 4.7 on page 72.

> LGOCV - when do we use it?

It depends. It has good variance properties if you do a good number of resamples and the bias is really dependent on what percentage of the training data gets left out. If you have a lot of computing power, this might be the preferred method.

> my Y variable is not hierarchical

Not sure what you mean.

Note that we call this LGOCV but we are only holding out a single sample (see the discussion in section 12.1). We needed to call it something in code.

> twoClassSummary - can it be used only when we have two classes?



  • $\begingroup$ thanks, i looked at the section 12.1. But didnt see any material describing LGOCV. any particular page that you are referring to? if we are holding only one sample then isn't it same as LOOCV? i looked at the code and a bit curious. why do we have to say "index = list(TrainSet = pre2008)"? i tried changing it with "index = list(pre2008)" but got errors :( $\endgroup$ Aug 29, 2013 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.