7
$\begingroup$

I'm using the R package caret to generate classifiers using a variety of different models on an imbalanced dataset. To overcome the class imbalance problem, I am using the "weights" parameter in the "train" function. This seems to work for some models, but not for others. If I set the weights equal, the rpart classifier, for example, predicts 20% class 1 and 80% class 2, which is very close to my observed class proportions. Setting the weights to 10 and 1, the classifier now predicts class 1 in over 90% of examples, and only 10% class 2.

However, when using the svmLinear, svmRadial, or rf algorithms, the weights parameter has no effect. No matter what class weights I input, the classifiers invariably label 20% as class 1 and 80% as class 2. Looking into the code a little more, it doesn't appear that the "weights" parameter from the "train" function ever gets passed through to the call to the ksvm package, which is what caret uses for SVM training.

This link has an example using the svmLinear method in caret, and changing the weight vector does absolutely nothing.

Has anyone else experienced this problem? Is there any way to determine which models actually accept class weights as an input?

$\endgroup$
3
  • $\begingroup$ If its just svm that gives you headaches you could also try the package e1071. The weighting works as in the docs: Edit: Damn, cant format correctly in comments. See the last example in ?svm. $\endgroup$ Aug 4, 2015 at 12:32
  • $\begingroup$ Now caret supports svm weights by kernlab github.com/topepo/caret/commit/… $\endgroup$
    – heroxbd
    May 19, 2016 at 6:30
  • $\begingroup$ For caret, to find out which models accept class weights, can go to github.com/topepo/caret/, search keywords: if(!is.null(wts)) theDots$weights <- wts. At this moment, 32 pieces of modelInfo return, which means there are 32 built-in caret models available to pass class weights. $\endgroup$
    – CcMango
    Oct 29, 2017 at 22:32

1 Answer 1

12
$\begingroup$

I haven't gotten around to implementing it for all the models that can accept weights. Right now, it should work for rpart variants, glmnet, gamSpline, glmboost, gamboost, evtree, ctree, ctree2, chaid, cforest, blackboost, treebag, glm, glmStepAIC, and bayesglm.

Note that ksvm function does not have a weight parameter, so those models won't be enabled.

$\endgroup$
4
  • $\begingroup$ Thanks for the info, it's great to hear directly from the source. I'm a little confused about your last sentence, I don't really know what you mean by a model being "enabled". I'm using the "class.weights" parameter, although I just noticed that the linked example I posted tries to use a parameter just called "weights", which I suppose doesn't exist. $\endgroup$ Jul 24, 2015 at 7:37
  • $\begingroup$ kernlab has weight parameter now and caret supports that. Please consider updating this answer. I was mislead at first. $\endgroup$
    – heroxbd
    May 19, 2016 at 6:31
  • 3
    $\begingroup$ Almost two years passed.Is there an updated version of this list? Thanks. $\endgroup$
    – Lior Kogan
    Jun 14, 2017 at 16:22
  • $\begingroup$ Perhaps check here: topepo.github.io/caret/available-models.html# There is a column that allows you to see what models have "weight" for a tuning parameter. $\endgroup$
    – Pake
    Dec 21, 2020 at 20:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.