3
$\begingroup$

i am learning data mining through book . During classification chapters about Neural Networks the authors have below code. I have below questions:

## pre2008 <- 1:nrow(training) ## training is a dataset that has training data
ctrl <- trainControl(method = "LGOCV",
                     summaryFunction = twoClassSummary,
                     classProbs = TRUE,
                     index = list(TrainSet = pre2008),
                     savePredictions = TRUE)
nnetGrid <- expand.grid(.size = 1:10, .decay = c(0, .1, 1, 2))
maxSize <- max(nnetGrid$.size)
set.seed(476)
nnetFit <- train(x = training[,reducedSet], 
                 y = training$Class,
                 method = "nnet",
                 metric = "ROC",
                 preProc = c("center", "scale"),
                 tuneGrid = nnetGrid,
                 trace = FALSE,
                 maxit = 2000,
                 MaxNWts = 1*(maxSize * (length(reducedSet) + 1) + maxSize + 1),
                 trControl = ctrl)

LGOCV - when do we use it? I read the post, but still not clear. the post says that it is a variant of LOOCV for hierarchical data. but my Y variable is not hierarchical :(

twoClassSummary - can it be used only when we have two classes? can i used it for say Iris data?


LGOCV is also known as Monte-Carlo Cross Validation. More details are available here.

$\endgroup$
  • $\begingroup$ A quick Google establishes "leave-group-out cross validation" as the answer to your first question. Other questions are all focused on software/programming and arguably off-topic here. $\endgroup$ – Nick Cox Aug 27 '13 at 17:30
  • $\begingroup$ the post that i have mentioned in my email says that LGOCV means "leave-group-out cross validation" , but i would like to know more details of it. for example: when we should use it and does it need Y to be hierarchical .. $\endgroup$ – user2543622 Aug 27 '13 at 17:56
  • 1
    $\begingroup$ OK, but you said "what does it stand for?" and I answered that. Perhaps you should make your question clearer. $\endgroup$ – Nick Cox Aug 27 '13 at 17:59
2
$\begingroup$

From the book: "Repeated training/test splits is also known as 'leave-group-out cross- validation' or 'Monte Carlo cross-validation.'". It is illustrated in Figure 4.7 on page 72.

> LGOCV - when do we use it?

It depends. It has good variance properties if you do a good number of resamples and the bias is really dependent on what percentage of the training data gets left out. If you have a lot of computing power, this might be the preferred method.

> my Y variable is not hierarchical

Not sure what you mean.

Note that we call this LGOCV but we are only holding out a single sample (see the discussion in section 12.1). We needed to call it something in code.

> twoClassSummary - can it be used only when we have two classes?

Yes.

Max

$\endgroup$
  • $\begingroup$ thanks, i looked at the section 12.1. But didnt see any material describing LGOCV. any particular page that you are referring to? if we are holding only one sample then isn't it same as LOOCV? i looked at the code and a bit curious. why do we have to say "index = list(TrainSet = pre2008)"? i tried changing it with "index = list(pre2008)" but got errors :( $\endgroup$ – user2543622 Aug 29 '13 at 17:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.