# SVM with unequal group sizes in training data

I am trying to build an SVM from training data where one group is represented more than the other. However, the groups will be equally represented in the eventual test data. Therefore, I'd like to use the class.weights parameter of the e1071 R package interface to libsvm to balance the influence of the two groups in the training data.

Since I was unsure exactly how these weights should be specified, I set up a little test:

1. Generate some null data (random features; 2:1 ratio between group labels)
2. Fit an svm with the class.weights parameter set.
3. Predict a bunch of new null datasets and look at the class proportions.
4. Replicate the whole process many times for different null training sets.

Here is the R code I'm using:

nullSVM <- function(n.var, n.obs) {
# Simulate null training data
vars   = matrix(rnorm(n.var*n.obs), nrow=n.obs)
labels = rep(c('a', 'a', 'b'), length.out=n.obs)
data   = data.frame(group=labels, vars)

# Fit SVM
fit = svm(group ~ ., data=data, class.weights=c(a=0.5, b=1))

# Calculate the average fraction of 'a' we would predict from null test data
mean(replicate(50, table(predict(fit, data.frame(matrix(rnorm(n.var*n.obs), nrow=n.obs))))[1])) / n.obs
}

library(e1071)
set.seed(12345)
mean(replicate(50, nullSVM(50, 300)))


From this whole thing I was expecting an output ~ 0.5, however, that's not what I got:

> mean(replicate(50, nullSVM(50, 300)))
[1] 0.6429987


The class.weights paramter is working, sort of, as the lower I weight a, the lower it is represented in this simulation (and if I omit class.weights it returns close to 1)...but I do not understand why simply using weights of 1:2 (for training data that is 2:1) does not get me all the way down to 50%.

If I'm misunderstanding SVMs, can someone explain this point? (or send some refs?)

If I'm doing it wrong, can someone tell me the correct way to use the class.weights parameter?

Could it possibly be a bug? (I think not, since I understand this software and the underlying libsvm to be quite mature)

• I don't have experience with libsvm but with LiblineaR, the class weights are crucial. Withouth setting it correctly, you get sub-optimal results if your classes are heavily unbalanced. I would suggest: Get a real dataset with unbalanced classes and try different values of class.weights (in LiblineaR wi). LiblineaR is orders of magnitude faster for a lineal kernel and has penalized methods also. In my experience, you first find a decent class weight and then optimize C. – marbel Oct 12 '14 at 21:59

• Of course on real data I always use the caret package or the built-in tune() function for model parameter tuning, so I especially like your second idea for how to deal with this in practice by adjusting the resampling scheme to favor the minority class. Much appreciated. – John Colby Nov 5 '11 at 17:08