I am interested in a multiple classes problem with imbalanced classes and I was rather happy with the caret package so far but, I have some practical and theoretical questions regarding the multiple classes context :

  • Is it possible to use another metric than accuracy and kappa, I don't find how to use F-measure or G-mean ?
  • When the sampling is done during resample, does this mean that for a 5 folds cross validation, we have two partitions, the original and a resampled one, and we use for each validation fold (drawn from the original partition), the four correspondant folds from the resampled partition ? (this is important when oversampling) (the question is valid for binary classifications too)
  • I've read that there are classic decomposition methods to do multiple classes classification using binary classifications (one per class or OpC also called one against all or OAA or OVA ; or pairwise classes binary classifications), but I don't get if we can alter these options when calling a method in train

Here is an example of my code for some method and its relevant grid.

myControl <- trainControl(method='cv', number=5, classProbs = TRUE,sampling='up')

tune <- train(data.train[,-1],data.train[,1],tuneGrid=grid,method = method,metric= 'Kappa',trControl =myControl)

closed as off-topic by Xi'an, gung - Reinstate Monica, Christoph Hanck, Reinstate Monica, conjugateprior Dec 8 '15 at 16:41

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – Xi'an, gung - Reinstate Monica, Christoph Hanck, Reinstate Monica, conjugateprior
If this question can be reworded to fit the rules in the help center, please edit the question.