2
$\begingroup$

I know what is SRM but I didn't understand the relation between SRM and SVMs. Can anyone explain me this? Why they say that SVMs rely on a SRM approach? Thank you so much!

$\endgroup$

2 Answers 2

1
$\begingroup$

Without context it is a bit hard to say what is meant by the phase. However, I take guess. For the sake of the argument, let's use use a linear SVM in a two dimensional case as shown in this example on Wikipedia:

H3 does not separate the classes. H1 does, but only with a small margin. H2 separates them with the maximum margin.

While planes H1 and H2 separate the training data correctly, H2 is chosen by the SVM because it has higher margin. The greater the size of this "buffer zone" the lower the risk of a wrong classification for an unseen sample. Hence, one might say that a SVM does SRM by choosing the separator with the greatest margin.

$\endgroup$
0
$\begingroup$

As you probably know, structural risk minimisation normally consists of two steps

  1. Minimise the empirical risk $R_\mathrm{emp}$ in each of the function classes
  2. Minimise the guaranteed risk $R_\mathrm{g} = R_\mathrm{emp} + \mathrm{complexity}$

The point now is that the margin can be seen as a measure of complexity. Figure 1 in a paper from Osuna et. al provides a good intuition as to why a bigger margin is similar to lower complexity and vice versa.

When training a support vector machine, you try to find the classification border that minimises the number of misclassifications and at the same time maximises the margin. In other words, you try to minimise $R_\mathrm{emp}$ together with the complexity (larger margin). Therefore you could argue that an SVM minimises $R_\mathrm{g}$ and thus is an example of SRM.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.