# libSVM for unbalanced data

I'm using libSVM for binary classification and my training data is very unbalanced (-1:90%, +1:10%). According to libSVM's documentation, it's better to set different penalties for positive and negative classes. For example, the SVM problem is:

$\min\limits_{w,b,\xi} \frac{1}{2}{\bf w^Tw} + C^+\sum\limits_{y_i=1} \xi_i + C^-\sum\limits_{y_i=-1} \xi_i$

My question is which penalty should be larger and why. Thanks

-
Check out this paper. It shines some light on ideas on how to deal with unbalanced data. –  nickb Jun 5 at 0:09
Welcome to the site, @nickb. Would you mind adding a brief summary of the information in that paper in case the link goes dead, &/or so readers can know if they want to pursue it further? –  gung Jun 5 at 0:30
The larger the penalty, the more an error on the training set (which is what is measured by $\xi_i$) for a pattern of that class influences the model. So if you have more negative patterns than positive patterns then you probably want to make $C^+$ larger than $C^-$. Personally if there is a class imbalance problem then it usually means that the costs of false-positive and false-negative errors are not the same, and the relative costs of the errors is an important criterion for adjusting the penalties. I would suggest using cross-validation to estimate the expected loss and choose the penalties to minimise that.