I am interested in a multiple classes problem with imbalanced classes and I was rather happy with the
caret package so far but, I have some practical and theoretical questions regarding the multiple classes context :
- Is it possible to use another metric than accuracy and kappa, I don't find how to use F-measure or G-mean ?
- When the sampling is done during resample, does this mean that for a 5 folds cross validation, we have two partitions, the original and a resampled one, and we use for each validation fold (drawn from the original partition), the four correspondant folds from the resampled partition ? (this is important when oversampling) (the question is valid for binary classifications too)
- I've read that there are classic decomposition methods to do multiple classes classification using binary classifications (one per class or OpC also called one against all or OAA or OVA ; or pairwise classes binary classifications), but I don't get if we can alter these options when calling a method in
Here is an example of my code for some method and its relevant grid.
myControl <- trainControl(method='cv', number=5, classProbs = TRUE,sampling='up') tune <- train(data.train[,-1],data.train[,1],tuneGrid=grid,method = method,metric= 'Kappa',trControl =myControl)