I'd like to run a grid search cross-validation on the probability outputs of the SVC classifier. In particular I'd like to minimize the negative log likelihood. Is this a reasonable thing to do? I'm doing it b/c the dataset is: a) imbalanced 5:1 and b) not at all separable i.e. even the "worst" observations have a > 50% chance of being in the "good" class.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Yes, it makes sense to do so, provided that you try the following:
1) Since the training labels are imbalanced, you can weight the training sample points based on their labels. For example, if you have 5:1 labels ratio, you can weight those lesser labels with 5 times the weight of the greater labels. Since you said SVC, I assume you are talking about python scikit sklearn's SVM-C implementation. You can simply set the parameter class_weight='auto'.
2) SVM is supposed to handle non-separable data points. Just tune the C parameter. It controls how much do you think it matters that a point appears on the wrong side of the hyperplane. If C is high, then the SVM concentrate more on reducing the distance of wrongly classified training points from the decision plane. If C is low, then the SVM concentrate more on maximizing the margins for the correctly classified training points. The default for sklearn.svm.SVC is C=1.0.
3) You could also try using a kernel. Either one of the common kernels, e.g. 'rbf', 'poly', 'sigmoid', etc. or your custom kernels. In your linear space, your assertion in b) might be true, but in the high (or infinite) dimensional inner product space of one of kernels, they might be linear separable or at least much less mixed up.
