As far as I know, Logistic Regression can deal with data in which positive and negative samples can be separated by a linear hyperplane. But if the data cannot be separated by a hyperplane, it cannot be learned by simple Logistic Regression, more features should be added to make a higher space in which the data can be well learned by linear hyperplane. My question is: How to determine whether a dataset is linear enough that no more features should be added?


You can have some interactions which could account for various nonlinearities.

Linear model means that outcome can be expressed as an linear function of parameters, variables can be cross-products, squares, cubes etc. So in the end linear models in reality allow flexible modeling.

For logistic regression outcome is logit and modeled class probability can be calculated via transformation of this logit by exponentiation and other operators.

I would suggest not to worry about linearity of dataset, whatever that means, but only checking if your model has power to classify outcomes for example by using ROC-analysis.

  • $\begingroup$ Do you mean that I just regard the data as separable and check the model just by ROC? $\endgroup$ – maple Dec 10 '14 at 6:21
  • $\begingroup$ @maple , yes I do. $\endgroup$ – Analyst Dec 10 '14 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.