How support vectors is calculated on SVM example? Question: If we map input data with following $\phi$ function to higher dimension via Hard Margin SVM then support vectors are $a$ and $b$.
How we can find support vectors of this example, i.e: calculating $a$ and $b$.

My solution: (I try this but not reached to solution of example as depicted in image).

Update: I think one of support vector is wrong. any idea?
Update: as one nice answer so none of them is SV in this example. as we know For the hard margin, the closest points to the separating hyperplane are the support vector so can we select one of these vector as SV's base on this criteria?
Update 2: see my solution now.

 A: Regarding the comment in the question:

The real question is that: for detecting that some vector is support vector we should use which of them? 1) check the wx+b or 2) calculate Euclidean distance to hyperplane?

We can do both. This is because the support vectors, as has been already said by @Jacques Wainer, are the ones that correspond to points lying in the minimum margins (hard margin case). So if we have an expression for the hyperplane (in feature space):
$$ f(\phi(x))=0$$
Then the points $\phi(x_i)$ that don't lie in this hyperplane will have a value $\hat{\gamma_i}=|f(\phi(x_i))|>0$, being greater as the distance between the hyperplane and the point $\phi(x_i)$ increases. This value $\hat{\gamma_i}$ is known as functional margin.
Hence, the support vectors, $x_{SV}$, will correspond to the points $x_i$ that in feature space , $\phi(x_i)$, satisfy:
$$x_{SV} = \arg \min_{x_i} |f(\phi(x_i))| = \arg \min_{x_i} \hat{\gamma_i}$$
Note that the absolute value $|f(\phi(x_i))|$ is used to take into account that the points $\phi(x_i)$ can be at both sides of the hyperplane i.e. $f(\phi(x_i))$ will be positive for the data points with an specific label, and negative for the data points with the other label.
So we can see here that using $f(\phi(x))$ is sufficient to know which points in feature space ($\phi(x_i)$) correspond to support vectors.
However, note that this is equivalent as finding the points $\phi(x_i)$ with the minimum Euclidean distance to the hyperplane. This distance is also known as geometric margin ($\gamma_i$) and can be calculated as $\gamma_i = \hat{\gamma_i}/\Vert w\Vert$. Where $w=(4,9,4,0)^T$ in your example.
So to sum up, support vectors in your example, would correspond to the the points $\phi(x_i)$ of your dataset that minimize Euclidean distance ($\gamma_i$, geometric margin)  to the hyperplane, or equivalently, the ones that minimize the value of $|f(\phi(x_i))|$ ($\hat{\gamma_i}$, functional margin).

Particularizing to your data points $a$ and $b$, we have that:
$$|f(\phi(a))| = \hat{\gamma_a}=17 \quad\quad\quad |f(\phi(b))| = \hat{\gamma_b}=9$$
Given this, we can conclude that only if the rest of the data points used to construct the hyperplane $f(\phi(x))=0$ have bigger or equal functional margins, then $b$ will be a support vector.
A: Let us use the figure on the wikipedia pare of SVM https://en.wikipedia.org/wiki/Support_vector_machine#/media/File:SVM_margin.png as a guide.

*

*I understand that the $f(\phi(x))$ formula is for the separating hyperplane which is the red line in the figure. The formula is really $f(\phi(x))=0$.

Then $w = [4,9*4,4,0]$ and the $x$ in the formula in the figure is really $\phi(x)$ and $b = 0$. (in one point you have $9*4 \phi(x)_2$ and on your written notes you have $9*\phi(x)_2$ - I will assume that  $9*4 \phi(x)_2$ is the correct one)
Thus the hyper-plane equation is $[4,9*4,4,0]*\phi(x) - 0 = 0$


*According to the figure, the margins are   $[4,4*9,4,0]*\phi(x) - 0 = \pm 1$
Expanding the +1 version:
$$ [4,4*9,4,0]*\phi(x) - 0 =  1 \\
 [4,4*9,4,0]*[x_1^2,x_1^2, x_1 x_2, -x_1] = 1 \\
 10*4 x_1^2 + 4 x_1 x_2 = 1\\
10 x_1^2 + x_1 x_2 = 1/4 
$$
and similarly for the other margin
$$10 x_1^2 + x_1 x_2 = - 1/4 $$


*none of the points $a = [1,1]$ and $b=[1,-1]$ satisfy either equations so they are not in the margins.


*So they cannot be support vectors. Notice that you cannot calculate/compute what are the support vectors. The support vectors are the points on the training set that lie on the two margins - the two blue and one green points in the figure that have the black borders. You know that the support vectors lie on the margins but you need the training set to select/verify the ones that are the support vectors.
UPDATE:
given that the correct formula for the hyperplane is the one without $9*4\phi(x)_2$

*

*the margins equations are $[4,9,4,0]*\phi(x) - 0 = \pm 1$


*For the +1 margin
$$ [4,9,4,0]*\phi(x) - 0 =  1 \\
 [4,9,4,0]*[x_1^2,x_1^2, x_1 x_2, -x_1] = 1 \\
 11 x_1^2 + 4 x_1 x_2 = 1\\
 11 x_1^2 + 4 x_1 x_2 - 1 = 0\\
$$
for the -1 margin
$$11 x_1^2 + 4 x_1 x_2 + 1 = 0$$


*again, neither a=[1,1] or b=[1,-1] are in the margins and therefore cannot be support vectors.


*There seems to be a problem with your derivation. There is no need to divide $Y_i$ by $||w||$.
$ 2/||w||$ is the size of the margin. But the equation of the hyperplane $f(x)$ is computer such that:

*

*$f(x) = 0 $ for X in the separating hyperplane

*$f(x) = 1 $ for one of the margins

*$f(x) = -1$ for the other margin

(please see again the figure in wikipedia).
