0
$\begingroup$

I am trying to understand the relationship between structural or statistical properties of training dataset and hardness of classification in the context of binary classification with SVM using RBF kernel. I would like to predict the hardness without actually trying to classify the dataset. The obvious properties are the size of the dataset and features. What other meta-properties indicate that an SVM using RBF kernel will result in low accuracy and/or extremely expensive computation?

$\endgroup$
  • $\begingroup$ I would look at the distribution of the distances. consider that the model depends not really on the features, but on the distance matrix. $\endgroup$ – carlo Sep 30 at 16:47
  • $\begingroup$ @carlo, could you clarify the definition of "distance"? May I know distance between what? $\endgroup$ – Omar Shehab Oct 1 at 14:17
  • $\begingroup$ RBF kernel uses distances $||x_1-x_2||$ $\endgroup$ – carlo Oct 1 at 15:16
0
+50
$\begingroup$

The idea of using properties of a dataset to decide on classifier characteristics is called metalearning

I do not know much about metalearning itself, nor any specific aspects of metalearning for SVMs and classification hardness

A search on Google Scholar points to https://link.springer.com/article/10.1007/s10462-013-9406-y (open access) as a recent and well cited survey of metalearning.

I know of one result on metalearning-like results for SVM: the mean distance between classes is a good heuristic value for the $\gamma$ hyperparameter for the RBF SVM (both behind paywall):

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

The RBF kernel is local in the feature space, so it can only work well if a nearest neighbour predictor also works fairly well. It is often worth trying Nearest Neighbour first - if its results are dreadful then I question whether I have the right feature set.

But if you are going to use an SVM, it does not feel right to me to start with feature engineering then use assume the RBF is appropriate. Instead, design a problem-relevant kernel function. This will be equivalent to some set of features, by Mercer's Theorem, but what that set is does not really matter - you get the predictions directly by building an SVM with your kernel function.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.