If I have more than 3 features in a dataset, then I can't visualize them to see if my classes are scattered in a non linear fashion. So how do I know when is the right way to use linear model with non-linear(polynomial) basis function in logistic regression or Support Vector Machines?. I just don't want to use a kernel function in SVM without knowing why that should be used.

Is there a way to know when to use Kernel functions of non linear basis functions in a linear model by looking at the data?

  • 3
    $\begingroup$ Possible duplicate of Linear kernel and non-linear kernel for support vector machine? $\endgroup$ Nov 13, 2015 at 20:54
  • 6
    $\begingroup$ I'm voting to leave open, as the OP's focus seems not on which to use, but on whether one can somehow tell which to use without running a linear kernel SVM. (It might well be that the answer is, "No, you pretty much have to run an SVM or another linear classifier," but the questions strike me as distinct.) $\endgroup$ Nov 13, 2015 at 21:24
  • $\begingroup$ By looking at the data, no. You should start with the linear model and plot learning curves (error as a function of the training set size). Plot both training and testing errors. If the both errors converge to some plateau and in absolute value are high, then you are in high bias setting - you need more powerful models, e.g. non-linear kernel SVM. Check out Andrew Ng's lecture about Learning Curves (class.coursera.org/ml-005/lecture/64). $\endgroup$ Nov 13, 2015 at 22:01
  • $\begingroup$ For completeness, if the gap between two curves is large and the training error is very low, you are in high variance or overfitting setting. Here some regularization and/or more data can help. $\endgroup$ Nov 13, 2015 at 22:03


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.