1
$\begingroup$

since I was reading about disadvantages of svm(support vector machine)

Non-Probabilistic - Since the classifier works by placing objects above and below a classifying hyperplane, there is no direct probabilistic interpretation for group membership. However, one potential metric to determine "effectiveness" of the classification is how far from the decision boundary the new point is.

it said that it has no direct probabilistic interpretation for the group
why this is disadvantages of the svm? for my thought this will increase the speed since it does not have to deal with the probabilistic interpretation for the group member.
but I might be wrong.
1. what is the actual meaning of 'no direct probabilistic interpretation?'
2. why this chracteristic is disadvantages of svm?
3. and what other method provide the 'probabilistic interpretation'?

$\endgroup$

1 Answer 1

2
$\begingroup$

Most machine learning methodologies do not directly vote on class membership. That is, when you ask most non-svm machine learning models for a prediction, they do not respond "this observation is an observation of class 0", or "this observation is an observation of class 1." Instead, they report a score, and the user may choose to create a classification rule by applying a threshold to the score.

Some models go a bit further and attempt to produce a score that is also a statistical estimate of the conditional probability:

$$ P(Y = 1 \mid X) $$

what is the actual meaning of 'no direct probabilistic interpretation?

An SVM makes no attempt to estimate the conditional probability $P(Y = 1 \mid X)$. It only attempts to produce class assignments.

and what other method provide the 'probabilistic interpretation'?

Logistic regression, gradient boosted classification, and neural networks have this property, and to a lesser extend so do both classification trees, and random forests.

why this characteristic is disadvantages of svm?

Models that produce an estimate of $P(Y = 1 \mid X)$ are useful in a wider set of circumstances than those that do not. For example, if you have a measurement that depends on $Y$ is some way, say:

$$ \text{Profit} = f(Y) $$

Then knowing the conditional probability $P(Y = 1 \mid X)$ is an important step to computing, say, the expected profit for a given set of circumstances $X$:

$$ E \left[ \text{Profit} \mid X \right] = f(0) P(Y = 0 \mid X) + f(1) P(Y = 1 \mid X) $$

If you only have a model that make as good guess at the class membership, you can't do this.

for my thought this will increase the speed since it does not have to deal with the probabilistic interpretation for the group member.

Maybe, maybe not. Even if that was so, model fitting speed is usually a secondary concern to model suitability. Since there are so many situations where having a handle on some probabilities is important, it very often outweighs any advantages of an SVM.

$\endgroup$
2
  • $\begingroup$ I would say anything trained with logloss error ( xgboost and neural networks) has probability output, tree /random forest classifier does not. Then you have to do probability calibration using platter scaling or isotonic regression. $\endgroup$
    – seanv507
    Commented Apr 3, 2019 at 5:27
  • $\begingroup$ The splitting criteria for trees can be viewed as greedy minimization of a loss function. The entropy criterion corresponds to log-loss: benkuhn.net/tree-imp. At the same time, they do tend to have poor calibration (which was the origin of my "to a lesser extent"). $\endgroup$ Commented Apr 3, 2019 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.