# Why is the equation for a single-neuron perceptron decision boundary Wp + b = 0 set to ZERO?

I am learning about artificial neural networks. I understand how the weights determine the slope of the (orthogonal) decision boundary and how the bias shifts that decision boundary, much like a line. What I do not understamd is why the equation is always set to zero?

That's linear algebra, not specific to perceptrons or machine learning. Let's start with a simple, two-dimensional case, with axes $$x_1$$ and $$x_2$$. Your class boundary can be defined by a straight line:

$$x_2 = a x_1 + b$$

This is a so-called explicit line equation. It has one problem: What if the boundary is a vertical line? In that case $$a$$ would need to be infinte. You can circumvent this problem by swapping the coordinates and writing:

$$x_1 = c x_2 + d$$

A vertical line is then described by $$x_1 = d$$, i.e. taking $$c=0$$. However, here you have the same problem with the horizontal line, because $$c$$ would have to be infinite. In the first equation, the horizontal line would be written down as $$x_2 = b$$, i.e. setting $$a$$ to zero.

The implicit equation, which is easy to derive from the above two, unifies the notation into a general form and is not susceptible to that problem:

$$w_1 x_1 + w_2 x_2 + b = 0$$

You can get the first explicit equation from it by setting $$w_1 = a$$ and $$w_2 = -1$$, and the second equation by setting $$w_1 = -1$$, $$w_2 = c$$, and $$b=d$$. A horizontal line can be written down by setting $$w_1 = 0$$, and a vertical one by setting $$w_2 = 0$$.

Using vector notation, with $$w = [w_1, w_2]$$ and $$x = [x_1, x_2]$$, the implicit equation can be written more compactly as:

$$w \cdot x + b = 0$$

This can be generalized to any number of dimensions.

• The simplified case helps me understand a bit more, but can you clarify how the implicit equation is not susceptible to the infinite a problem? Is it not possible to have a verticle line as a decision boundary? or is it becuase the if the wieght is infinite then the decision boundary is horizontal? Jan 29, 2020 at 18:46
• I edited the answer and hope it to be clearer now. Jan 30, 2020 at 9:28
• Could you please elaborate more on the explicit line equations? I couldn't get why a would need to be infinite. Is it because a would need to compensate for all the different values of $x_2$ regardless of $b$ being any arbitrary value? Dec 21, 2020 at 10:20
• @AndréYuhai $a$ is the slope of the line. It gives you the change in $x_2$ for a unit change in $x_1$. For $a=1$, if you change $x_1$ by 1, $x_2$ will also change by 1. That's a 45° line. As $a$ grows, the line becomes steeper. For $a=1000$, increasing $x_1$ by 1 causes an increase of $x_2$ by 1000. That's already much steeper, but still not exactly vertical. Only when $a = \infty$, the line becomes vertical. It's easy to understand if you draw it up. Dec 22, 2020 at 9:54
• Oh, now I got it! Thank you! Dec 22, 2020 at 13:08

The equation is parametrizing a hyperplane.

You could set your decision boundary to be any parametrized by say Wp + b' = 1 if you wanted to. However, the results would be the same. Since this would be equal to saying the decision boundary is Wp + b'-1 = 0, and then we can define b=b'-1 and estimate b instead of b'