I have a question regarding the definition of support vectors. It is usually stated that support vectors are those vectors which lie on the hyperplanes and hence define them.
But how is the green point, which lies on the hyperplane H1, spanned for sperating the red points, termed?
Is it also a support vector since it lies on a hyperplane or is it simply a misclassified point which does not support the hyperplane H1?
(The purple circles shall represent the support vectors)
It's a misclassified point. What you're looking is a SVM that has a soft margin, which can let a couple of instances on the wrong side of the hyperplane.
If a SVM were trained on this dataset it would continuously iterate until it gets a separating hyperplane, but these points can't be linearly separated, so training wouldn't stop. To fix this we add in a soft margin variable which adds in a little slack and will let it have a few instances on the wrong side of it's hyperplane.
Just a word - if the soft margin variable (slack penalty C) is equal to 0 it will ignore the data entirely.
