My understanding of SVM is that it's very similar to a logistic regression (LR), i.e. a weighted sum of features is passed to the sigmoid function to get a probability of belonging to a class, but instead of the cross-entropy (logistic) loss function, training is performed using the hinge loss. The benefit of using the hinge loss is that one can do various numerical tricks to make kernelisation more efficient. A drawback, however, is that the resulting model has less information than a corresponding LR model could have. So, for example, without kernelisation (using a linear kernel) the SVM decision boundary would still be at the same location where LR would output a probability of 0.5, BUT one cannot tell how quickly the probability of belonging to a class decays away from the decision boundary.
My two questions are:
- Is my interpretation above correct?
- How does using the hinge loss make it invalid to interpret SVM results as probabilities?