# Why perceptron is linear classifier?

It is said that perceptron is linear classifier, but it has a non-linear activation function f = 1 if wx - b >= 0 and f = 0 otherwise

If i will use some non-linear function on linear combination of my data, i think i will get a non-linear classifier. Why it is false?

It is called a linear classifier because its decision boundary is given by a (linear) hyperplane. Such a hyperplane is given by the set $$\{ x | w^tx =b \}$$ which thus splits $$\mathbf R^n$$ into two classes, $$\{x | w^tx \leq b\}$$ and $$\{ x | w^tx > b\}$$. You can think of $$w$$ as the normal vector to this hyperplane and $$b$$ as an offset by which you shift the hyperplane.

• Is a last layer of neural network a linear classifier? – mike May 31 '19 at 8:33
• That depends on your neural network, in general that does not have to be the case. – Stefan May 31 '19 at 8:39
• The hyperplane comment is the correct reason, but the first one isn't sufficient and may not lead to linear classification for certain activation functions – gunes May 31 '19 at 8:46
• @gunes: I think that depends on your definition of "linear classifier". Wikipedia (en.wikipedia.org/wiki/Linear_classifier) defines a linear classifier as a classifier that makes "decision[s] based on the value of a linear combination" of your data. – Stefan May 31 '19 at 8:53
• @gunes: I see your point, to me the wikipedia article is ambiguous, i.e. it does not rigorously define what a linear classifier is. Having looked up a more reliable source (Wasserman's All of Statistics) I'll agree with you that the correct definition of a linear classifier is "a classifier such that the decision boundary is a hyperplane" and not "a classifier that only depends on a linear combination of the inputs". I'll edit my post accordingly :) – Stefan May 31 '19 at 11:08

The activation function of a perceptron is linear in it's parameters, i.e. the decision boundary is defined by a linear function of the form $$w^tx = b$$ = $$\sum_{i} w_{i} * x_{i}$$.

Similar to linear least squares, the perceptron does not care whether the input $$x_{i}$$'s are non-linear combinations of your original data. You can indeed model a non-linear decision boundary using a polynomial basis function, e.g. if your data consist of a single independent variable $$x$$ you could include the square of $$x$$ as an additional variable.

Again, the perceptron can only model a linear decision boundary, but linear decision boundaries in quadratic space become a non-linear curve when mapped back to the original space.

• I dont think this answers this question correctly. While the sumproduct is calculated with linear function, the activation is a nonlinear function (like Sigmoid, ReLU , tan() and others) , so, since sigmoid is the output and it is nonlinear, then the perceptron , even if it is a single one becomes a nonlinear function. If you are saying that perceptron is linear, then show a proof. – Nulik Apr 8 '20 at 8:55
• @Nulik You have a point, but I'm convinced that as long as the activation function is monotonic, choosing a threshold on the activation will lead to the input being separated by a single hyperplane, as there is a bijective mapping between the outputs of the linear model and the non-linear activation. However, if the activation is non-monotonic (I tried this using the activation f(x) = abs(x)), there is a surjective-only mapping of the linear outputs onto the activations, so there are two (or more?) separating hyperplanes in input space, which I'd say would be a non-linear classifier. – Scholar Jun 20 '20 at 13:18
• @Nulik In any case, all separating lines are linear hyperplanes. – Scholar Jun 20 '20 at 13:20