Return to Answer

First, for support vectors the decision boundaries are given by $\omega^{*T}x^{(i)} + b = \pm 1$, and $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane.

The closest positive and negative examples to the separating hyperplane are,

$argmax_{i:y^{(i)} = -1} \omega^{*T}x^{(i)}$, resp. $argmin_{i:y^{(i)} = 1} \omega^{*T}x^{(i)}$

These verify (because the must be support vectors) the equations for the decision boundaries, that is,

$max_{i:y^{(i)} = -1} \omega^{*T}x^{(i)} + b = -1$, resp. $min_{i:y^{(i)} = 1} \omega^{*T}x^{(i)} + b = 1$

Add the two and solve for $b$.

P.S. why is $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane? We could solve it with some algebra (like here), or as a optimization problem :) The distance to a line is the norm of the vector point closest to the origin. That is, we would like to solve,

$$ min ||x||^{2} $$ subject to $\omega^{T}x + b = 0$. By introducing Lagrange multipliers we get, $$ L = \frac{1}{2}||x||^{2}-\lambda(\omega^{T}x + b) $$ If we derive with respect to $x$, equal to zero and solve for $x$, we get $x=\lambda \omega$. Subtitute back in the contraint and find $\lambda = \frac{-b}{||\omega||^{2}}$.

First, for support vectors the decision boundaries are given by $\omega^{*T}x^{(i)} + b = \pm 1$, and $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane.

The closest positive and negative examples to the separating hyperplane are,

$argmax_{i:y^{(i)} = -1} \omega^{*T}x^{(i)}$, resp. $argmin_{i:y^{(i)} = 1} \omega^{*T}x^{(i)}$

These verify (because the must be support vectors) the equations for the decision boundaries, that is,

$max_{i:y^{(i)} = -1} \omega^{*T}x^{(i)} + b = -1$, resp. $min_{i:y^{(i)} = 1} \omega^{*T}x^{(i)} + b = 1$

Add the two and solve for $b$.

P.S. why is $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane? We could solve it with some algebra (like here), or as a optimization problem :) The distance to a line is the norm of the vector point closest to the origin. That is, we would like to solve,

$$ min ||x||^{2} $$ subject to $\omega^{T}x + b = 0$. By introducing Lagrange multipliers we get, $$ L = \frac{1}{2}||x||^{2}-\lambda(\omega^{T}x + b) $$ If we derive with respect to $x$, equal to zero and solve for $x$, we get $x=\lambda \omega$. Subtitute back in the constraint and find $\lambda = \frac{-b}{||\omega||^{2}}$.

First, for support vectors the decision boundaries are given by $\omega^{*T}x^{(i)} + b = \pm 1$, and $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane.

The closest positive and negative examples to the separating hyperplane are,

$argmax_{i:y^{(i)} = -1} \omega^{*T}x^{(i)}$, resp. $argmin_{i:y^{(i)} = 1} \omega^{*T}x^{(i)}$

These verify (because the must be support vectors),

$max_{i:y^{(i)} = -1} \omega^{*T}x^{(i)} + b = -1$, resp. $min_{i:y^{(i)} = 1} \omega^{*T}x^{(i)} + b = 1$

Add the two and solve for $b$.

P.S. why is $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane? We could solve it with some algebra (like here), or as a optimization problem :) The distance to a line is the norm of the vector point closest to the origin. That is, we would like to solve,

$$ min ||x||^{2} $$ subject to $\omega^{T}x + b = 0$. By introducing Lagrange multipliers we get, $$ L = \frac{1}{2}||x||^{2}-\lambda(\omega^{T}x + b) $$ If we derive with respect to $x$, equal to zero and solve for $x$, we get $x=\lambda \omega$. Subtitute back in the contraint and find $\lambda = \frac{-b}{||\omega||^{2}}$.

First, for support vectors the decision boundaries are given by $\omega^{*T}x^{(i)} + b = \pm 1$, and $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane.

The closest positive and negative examples to the separating hyperplane are,

$argmax_{i:y^{(i)} = -1} \omega^{*T}x^{(i)}$, resp. $argmin_{i:y^{(i)} = 1} \omega^{*T}x^{(i)}$

These verify (because the must be support vectors) the equations for the decision boundaries, that is,

$max_{i:y^{(i)} = -1} \omega^{*T}x^{(i)} + b = -1$, resp. $min_{i:y^{(i)} = 1} \omega^{*T}x^{(i)} + b = 1$

Add the two and solve for $b$.

P.S. why is $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane? We could solve it with some algebra (like here), or as a optimization problem :) The distance to a line is the norm of the vector point closest to the origin. That is, we would like to solve,

$$ min ||x||^{2} $$ subject to $\omega^{T}x + b = 0$. By introducing Lagrange multipliers we get, $$ L = \frac{1}{2}||x||^{2}-\lambda(\omega^{T}x + b) $$ If we derive with respect to $x$, equal to zero and solve for $x$, we get $x=\lambda \omega$. Subtitute back in the contraint and find $\lambda = \frac{-b}{||\omega||^{2}}$.

First, for support vectors the decision boundaries are given by $\omega^{*T}x^{(i)} + b = \pm 1$, and $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane.

The closest positive and negative examples to the separating hyperplane are,

$argmax_{i:y^{(i)} = -1} \omega^{*T}x^{(i)}$, resp. $argmin_{i:y^{(i)} = 1} \omega^{*T}x^{(i)}$

These verify (because the must be support vectors),

$max_{i:y^{(i)} = -1} \omega^{*T}x^{(i)} + b = -1$, resp. $min_{i:y^{(i)} = 1} \omega^{*T}x^{(i)} + b = 1$

Add the two and solve for $b$.

First, for support vectors the decision boundaries are given by $\omega^{*T}x^{(i)} + b = \pm 1$, and $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane.

The closest positive and negative examples to the separating hyperplane are,

$argmax_{i:y^{(i)} = -1} \omega^{*T}x^{(i)}$, resp. $argmin_{i:y^{(i)} = 1} \omega^{*T}x^{(i)}$

These verify (because the must be support vectors),

$max_{i:y^{(i)} = -1} \omega^{*T}x^{(i)} + b = -1$, resp. $min_{i:y^{(i)} = 1} \omega^{*T}x^{(i)} + b = 1$

Add the two and solve for $b$.

P.S. why is $\frac{-b}{||\omega||}$ is the distance from the origin to the hyperplane? We could solve it with some algebra (like here), or as a optimization problem :) The distance to a line is the norm of the vector point closest to the origin. That is, we would like to solve,

$$ min ||x||^{2} $$ subject to $\omega^{T}x + b = 0$. By introducing Lagrange multipliers we get, $$ L = \frac{1}{2}||x||^{2}-\lambda(\omega^{T}x + b) $$ If we derive with respect to $x$, equal to zero and solve for $x$, we get $x=\lambda \omega$. Subtitute back in the contraint and find $\lambda = \frac{-b}{||\omega||^{2}}$.