0
$\begingroup$

I think linear model is such as linear regression, logistic regression (including one with polynomial features), SVM (including one with kernel).

I think non-linear model is such as decision tree, random forest, neural network, kNN.

Why do we need non-linear model, though we can deal with non-linear separable problem, using linear model (logistic regression, SVM etc, with polynomial feature or kernel trick)?

Polynomial feature strategy is computationally less effective than non-linear model? Is it also likely to cause the curse of dimension? That is, is polynomial feature strategy more likely to cause over-fitting than non-linear models?

$\endgroup$
  • 3
    $\begingroup$ Why use screwdriver if you have a hammer? $\endgroup$ – Tim Aug 25 '17 at 7:43
  • $\begingroup$ @Tim's comment is fun, but of course we need some explanation how the screwdriver is better suited for certain problems than a hammer. The polynomial feature strategy being less effective than the use of nonlinear models is one such argument. $\endgroup$ – Richard Hardy Aug 25 '17 at 8:03
0
$\begingroup$

Increasing dimensionality with polynomial features will hurt performance, or even make storing data infeasible if you start with too high dimension.

Kernel trick is strictly more powerful than adding finitely many features: for example using rbf kernel enables you to perform implicitly learning in an infinite-dimensional space.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.