Timeline for Support Vector Machine with Perceptron Loss
Current License: CC BY-SA 4.0
18 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 21, 2021 at 18:37 | comment | added | Igor F. | @Germania I see what you mean, but, again, it's a matter of definition. Cristianini and Shawe-Taylor in "An Introduction to Support Vector Machines...", p. 95 define the margin as 1/2 the distance between the closest vectors of the different classes, or the distance between the boundary and these vectors. Vapnik's Fig. 5.2 suggests that he uses the same convention, as does Bishop, PRML, Fig. 7.1. But I acknowledge that some researchers use the double value and that it may look more intuitive. For eventual computations in SVM it makes no difference. | |
| Oct 21, 2021 at 7:45 | comment | added | user318514 | margin=2/||w|| is correct. | |
| Oct 21, 2021 at 7:02 | history | edited | Igor F. | CC BY-SA 4.0 |
Corrected the margin.
|
| Oct 21, 2021 at 7:01 | history | rollback | Igor F. |
Rollback to Revision 1
|
|
| Oct 21, 2021 at 6:51 | comment | added | Igor F. | @Germania You probably mean the objective function, but no, it's not wrong. People use both versions, even in a same publication (see e.g. Vapnik, "The Nature of Statistical Learning", (5.9) vs. (5.10)), and Shalev-Schwartz and Ben-David in "Understanding Machine Learning" formulate it in terms of a general parameter $\lambda$ (15.6). You need to dive a bit into the math to understand why. | |
| S Oct 21, 2021 at 4:35 | history | suggested | user318514 | CC BY-SA 4.0 |
fixed objective function, I corrected the margin.
|
| Oct 20, 2021 at 23:04 | comment | added | user318514 | Your margins are wrong. | |
| Oct 20, 2021 at 22:28 | review | Suggested edits | |||
| S Oct 21, 2021 at 4:35 | |||||
| Oct 20, 2021 at 21:40 | comment | added | user318514 | You did not prove how the last two photos follows from the different inequality constraints involving xi. | |
| Jan 28, 2020 at 17:59 | vote | accept | Cagdas Ozgenc | ||
| Jan 28, 2020 at 17:58 | history | bounty ended | Cagdas Ozgenc | ||
| Jan 28, 2020 at 8:38 | comment | added | Igor F. | The VC-Theory gives you upper bounds on errors independent of the probability distribution behind the data. Of course, if you know the distribution, you might achieve a lower error using a more specific algorithm. E.g. if you have two Gaussian classes of the same size and variance, you are probably better off with logistic regression. But, in practice, you seldom know the distribution and, even if you did, the assumptions for an alternative known algorithm (in our case: two classes, equal size, equal variances) would probably be violated. | |
| Jan 27, 2020 at 20:34 | comment | added | Cagdas Ozgenc | You answered the original question I am not challenging your answer. Since you evangelized the max margin, I thought maybe you know a proof for minimax bound. For example two gaussians with equal diagonal covariances placed far apart from each other will be estimated quite poorly by max margin compared to simply taking the mid point of the means. For this reason I wanted to understand at least if there is a minimax defense for it. | |
| Jan 27, 2020 at 20:29 | comment | added | Igor F. | I am under the impression that you are "moving the goalposts". Originally, you wanted to know whether there is a theoretical reason for not using the "perceptron" loss function. I answered that. For everything else, I suggest you post a new, separate question. | |
| Jan 27, 2020 at 19:47 | comment | added | Igor F. | Also, if you take a look at the figures, maybe you can recognize that both lines (and infinitely many more) satisfy the "perceptron" condition: They correctly classify all points. But, are they all equally good class boundaries? | |
| Jan 27, 2020 at 19:30 | comment | added | Igor F. | See for example Nello Cristianini & John Shawe-Taylor: "Support Vector Machines", Cambridge University Press, 2000, p. 63 (quote of the theorem, originally, I believe, by Vapnik). | |
| Jan 27, 2020 at 18:45 | comment | added | Cagdas Ozgenc | Your reasoning about the optimization is valid. Having max margin vs another boundary is in my mind dubious, no matter how visually pleasing or intuitive it is. Can you provide a reference showing that max margin is actually a minimax boundary under all probability distributions for two class scenario? | |
| Jan 27, 2020 at 14:46 | history | answered | Igor F. | CC BY-SA 4.0 |