I know the normal equation is the closed form way of solving the linear regression problem. But how are they related to support vector machines?
I was asked this in an interview and did not know the answer.
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
If you have a point $p_0$ on a plane, and a vector $v$ which is orthogonal to the plane, then the plane can be expressed as the set of all points $p$ such that $v^T(p-p_0) = 0$. The goal of SVM is to find such a $v$ and a $p_0$ which maximizes the margin.
In fact, if $v$ is constrained to be a unit vector, then $|v^T(p-p_0)|$ is exactly the distance of point $p$ from the plane. Therefore the objective of SVM can be stated as finding the vector $v$ and $p_0$ such that all positive datapoints $p$ have
$$v^T(p-p_0) > \epsilon$$
and all negative datapoints $p$ have
$$v^T(p-p_0) < -\epsilon$$
for the largest possible $\epsilon$.
So to answer your question in a comment, $v^T(p-p_0)$ is the normal equation or normal form for a plane, which is different from what is called the normal equation for solving linear regression. It is also not the case that this is the normal equation for SVM -- it just happens to appear in the mathematical formulation of the SVM objective.