10
$\begingroup$

for eg. if I have a class variable credit scoring with two classes good and bad, where #(good) = 700 and #(bad)= 300. I do not want to shorten my data. which technique I should use? I was using SVM but it is giving all bad as good in predictions.

$\endgroup$
2
  • $\begingroup$ This asymmetry does not look so bad; are you sure you don't have an error somewhere else? Also can you give some more details about the data? There is not a general solution to this. $\endgroup$
    – user88
    Aug 26 '10 at 7:05
  • $\begingroup$ my dataset have 17 predictors(3 continuous and 14 categorical), obviously 1 class variable, and total 1000 obsrvations. The frequency dist. for class var. with train and test is train(bad) = 197, test(bad)= 103, trian(good)= 446, test(good)= 254 $\endgroup$
    – asad.taj
    Aug 26 '10 at 15:57
6
$\begingroup$

Your class sample sizes do not seem so unbalanced since you have 30% of observations in your minority class. Logistic regression should be well performing in your case. Depending on the number of predictors that enter your model, you may consider some kind of penalization for parameters estimation, like ridge (L2) or lasso (L1). For an overview of problems with very unbalanced class, see Cramer (1999), The Statistician, 48: 85-94 (PDF).

I am not familiar with credit scoring techniques, but I found some papers that suggest that you could use SVM with weighted classes, e.g. Support Vector Machines for Credit Scoring: Extension to Non Standard Cases. As an alternative, you can look at boosting methods with CART, or Random Forests (in the latter case, it is possible to adapt the sampling strategy so that each class is represented when constructing the classification trees). The paper by Novak and LaDue discuss the pros and cons of GLM vs Recursive partitioning. I also found this article, Scorecard construction with unbalanced class sizes by Hand and Vinciotti.

$\endgroup$
1
10
$\begingroup$

A popular approach towards solving class imbalance problems is to bias the classifier so that it pays more attention to the positive instances. This can be done, for instance, by increasing the penalty associated with misclassifying the positive class relative to the negative class. Another approach is to preprocess the data by oversampling the majority class or undersampling the minority class in order to create a balanced dataset.

However, in your case, class imbalancing don't seem to be a problem. Perhaps it is a matter of parameter tuning, since finding the optimal parameters for an SVM classifier can be a rather tedious process. There are two parameters for e.g. in an RBF kernel: $C$ and $\gamma$. It is not known beforehand which $C$ and $\gamma$ are best for a given problem; consequently some kind of model selection (parameter search) must be done.

In the data preprocessing phase, remember that SVM requires that each data instance is represented as a vector of real numbers. Hence, if there are categorical attributes, it's recommended to convert them into numeric data, using m numbers to represent an m-category attribute (or replacing it with m new binary variables).

Also, scaling the variables before applying SVM is crucial, in order to avoid attributes in greater numeric ranges dominating those in smaller numeric ranges.

Check out this paper.

If you're working in R, check out the tune function (package e1071) to tune hyperparameters using a grid search over supplied parameter ranges. Then, using plot.tune, you can see visually which set of values gives the smaller error rate.

There is a shortcut around the time-consuming parameter search. There is an R package called "svmpath" which computes the entire regularization path for a 2-class SVM classifier in one go. Here is a link to the paper that describes what it's doing.

P.S. You may also find this paper interesting: Obtaining calibrated probability estimates

$\endgroup$
1
  • 2
    $\begingroup$ +1; still ksvm from kernlab seems to be the preferred R SVM implementation. For instance it scales variables on its own and has a nice (=working very well) heuristic procedure for picking good $\gamma$ for RBF. $\endgroup$
    – user88
    Aug 26 '10 at 10:49
1
$\begingroup$

I would advise using a different value of the regularisation parameter C for examples of the positive class and examples of the negative class (many SVM packages support this, and in any case it is easily implemented). Then use e.g. cross-validation to find good values of the two regularisation parameters.

It can be shown that this is asypmtotically equivalent re-sampling the data in a ratio determined by C+ and C- (so there is no advantage in re-sampling rather than re-weighting, they come to the same thing in the end and weights can be continuous, rather than discrete, so it gives finer control).

Don't simply choose C+ and C- to give a 50-50 weighting to positive and negative patterns though, as the stength of the effect of the "imbalances classes" problem will vary from dataset to dataset, so the strength of the optimal re-weighting cannot be determined a-priori.

Also remember that false-positive and false-negative costs may be different, and the problem may resolve itself if these are included in determining C+ and C-.

It is also worth bearing in mind, that for some problems the Bayes optimal decision rule will assign all patterns to a single class and ignore the other, so it isn't necessarily a bad thing - it may just mean that the density of patterns of one class is everywhere below the density of patterns of the other class.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.