0
$\begingroup$

I'm trying to build a text classification model with SVM. The training data set consists of 100 string records with a one-to-one mapped response variable which is also a string. I can't split the data into training and test sets because there are 100 different classes in the response variable.

When I try to predict a new input, the model returns a response, but my question is is there any way to quantify this? Why I want to do this is if the model sees a totally new input, then based on some threshold on this metric, I can write some code to give feedback like "Can't predict as it is not trained before."

$\endgroup$
0
$\begingroup$

Since SVM returns an essentially unbounded distance value it can be hard to quantify that in easy terms like one can with probability. Ideally if you could do some cross validation you could look at the average max distance value returned across the different hold out sets and use that as your boundary for saying you're unable to predict. If you can't do any hold out sets you could a similar thing with your training set but set the threshold based on some mis-classification rate. I would however be concerned with over-fitting if you went that route.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.