I understood that increasing the dimensionality in SVM will help, but I try to understand the concept mathematically, as described in Breiman, Leo. "Statistical modeling: The two cultures." Statistical science 16.3 (2001): 199-231.

However, let us increase the dimensionality by adding as additional predictor variables all quadratic monomials in the original predictor variables; that is, all terms of the form $x_{m1}x_{m2}$. A hyperplane in the original variables plus quadratic monomials in the original variables is a more complex creature. The possibility of separation is greater. If no separation occurs, add cubic monomials as input features. If there are originally 30 predictor variables, then there are about 40,000 features if monomials up to the fourth degree are added.

So let's say I have $y = x +1$, then I do increase the dimensionality, how does this help with SVM?


If you have $y = x + 1$ then you don't even need statistical models, much less heavy guns such as support vector machines. And you can't have quadratic monomials with only one x variable.

For Breiman's comment to make any sense there need to be at least two independent variables (and usually more). For this sort of modeling to be necessary, the relationship between the DV and the IVs must be complex.

If a straight line fits your data very well, then why look for hyperplanes?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.