1
$\begingroup$

As we know (Wikipedia Definition): Linear Classifier makes a classification decision based on the linear combination of the feature vectors.
Mathematically : $y = f(\sum w_i x_i)$
So , $f$ is our linear classifier (which may be logistic or any other function). Now this linear classifier creates a decision boundary.

Now, for example consider only two features(X1, X2) : If the decision boundary is straight line then we say its linear decision boundary otherwise non linear decision boundary.

So, my question :
(1) If a classifier is a linear then it creates a linear decision boundary and vice versa.
(2) Non linear classifier always creates non linear decision boundary

Does the above statements are true if not then please explain? I have seen so many examples , like for SVM classifier, we transform the data to higher dimension and get the hyperplane in feature space but in input space it has non linear decision boundary.
So, what is the exact relation between a classifier and decision boundary, especially in the linear case?

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ Neither statement is generally true. The subtlety is that a nonlinear classifier may, by accident, create a linear boundary. Thus, the appearance of a linear boundary in one specific application does not determine whether the classifier as a procedure is linear or not. Also, the appearance of a linear manifold at some point in executing a procedure does not necessarily make the entire procedure a linear one. $\endgroup$
    – whuber ♦
    Jul 6, 2021 at 15:16
  • $\begingroup$ @whuber I got your explanation. So its all depend on the situations , both linear and non linear classifier can create any type of decision boundaries. Even thought logistic regression can create non linear decission boundaries too( if extra features are added) . Pls correct me if some thing is not true. $\endgroup$
    – Girish Kumar Chandora
    Jul 6, 2021 at 15:31
  • $\begingroup$ @whuber Pls refer to these question, the explanations are conflicting. Still I am not able to understand the concept of linear classifier. How do we know a classifier is linear ? $\endgroup$
    – Girish Kumar Chandora
    Jul 7, 2021 at 4:13
  • $\begingroup$ The answer you reference is a little sloppy because it lacks a suitable quantifier. You have to understand it as meaning all boundaries are linear, no matter what the inputs might be. $\endgroup$
    – whuber ♦
    Jul 7, 2021 at 13:49
  • $\begingroup$ @whuber Your time and help would be highly appreciated if you share any blog or reading material through which I gain these concept- Linear and Non Linear Classifier . Linear and Non Decision Boundaries. Even today I asked the question in different style on Data Science Community, but didnt get response. Have a look on question $\endgroup$
    – Girish Kumar Chandora
    Jul 7, 2021 at 13:56

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.