# How is normal equation related to support vector machines?

I know the normal equation is the closed form way of solving the linear regression problem. But how are they related to support vector machines?

I was asked this in an interview and did not know the answer.

• What are "normal equations"? – Harald Thomson Jul 23 '17 at 10:37
• Could it be possible that the interviewer intended "normal equation" to mean the parameterization of a hyper-plane by its normal vector? – shimao Jul 23 '17 at 17:04
• @shimao You are right, he said normal equation not equations. Could you elaborate a little bit more please? – LoveMeow Jul 23 '17 at 20:41
• If you have a point $p_0$ on a plane, and a vector $v$ which is orthogonal to the plane, then the plane can be expressed as the set of all points $p$ such that $v^T(p-p_0) = 0$. The goal of SVM is to find such a $v$ and a $p_0$ which maximizes the margin. – shimao Jul 23 '17 at 20:44
• @shimao could you write that as an answer so I can accept it? So this equation is the normal equation for an svm? – LoveMeow Jul 23 '17 at 20:46

If you have a point $p_0$ on a plane, and a vector $v$ which is orthogonal to the plane, then the plane can be expressed as the set of all points $p$ such that $v^T(p-p_0) = 0$. The goal of SVM is to find such a $v$ and a $p_0$ which maximizes the margin.

In fact, if $v$ is constrained to be a unit vector, then $|v^T(p-p_0)|$ is exactly the distance of point $p$ from the plane. Therefore the objective of SVM can be stated as finding the vector $v$ and $p_0$ such that all positive datapoints $p$ have

$$v^T(p-p_0) > \epsilon$$

and all negative datapoints $p$ have

$$v^T(p-p_0) < -\epsilon$$

for the largest possible $\epsilon$.

So to answer your question in a comment, $v^T(p-p_0)$ is the normal equation or normal form for a plane, which is different from what is called the normal equation for solving linear regression. It is also not the case that this is the normal equation for SVM -- it just happens to appear in the mathematical formulation of the SVM objective.