I am learning about artificial neural networks. I understand how the weights determine the slope of the (orthogonal) decision boundary and how the bias shifts that decision boundary, much like a line. What I do not understamd is why the equation is always set to zero?


2 Answers 2


That's linear algebra, not specific to perceptrons or machine learning. Let's start with a simple, two-dimensional case, with axes $x_1$ and $x_2$. Your class boundary can be defined by a straight line:

$$x_2 = a x_1 + b$$

This is a so-called explicit line equation. It has one problem: What if the boundary is a vertical line? In that case $a$ would need to be infinte. You can circumvent this problem by swapping the coordinates and writing:

$$x_1 = c x_2 + d$$

A vertical line is then described by $x_1 = d$, i.e. taking $c=0$. However, here you have the same problem with the horizontal line, because $c$ would have to be infinite. In the first equation, the horizontal line would be written down as $x_2 = b$, i.e. setting $a$ to zero.

The implicit equation, which is easy to derive from the above two, unifies the notation into a general form and is not susceptible to that problem:

$$w_1 x_1 + w_2 x_2 + b = 0$$

You can get the first explicit equation from it by setting $w_1 = a$ and $w_2 = -1$, and the second equation by setting $w_1 = -1$, $w_2 = c$, and $b=d$. A horizontal line can be written down by setting $w_1 = 0$, and a vertical one by setting $w_2 = 0$.

Using vector notation, with $w = [w_1, w_2]$ and $x = [x_1, x_2]$, the implicit equation can be written more compactly as:

$$w \cdot x + b = 0$$

This can be generalized to any number of dimensions.

  • $\begingroup$ The simplified case helps me understand a bit more, but can you clarify how the implicit equation is not susceptible to the infinite a problem? Is it not possible to have a verticle line as a decision boundary? or is it becuase the if the wieght is infinite then the decision boundary is horizontal? $\endgroup$ Jan 29, 2020 at 18:46
  • $\begingroup$ I edited the answer and hope it to be clearer now. $\endgroup$
    – Igor F.
    Jan 30, 2020 at 9:28
  • $\begingroup$ Could you please elaborate more on the explicit line equations? I couldn't get why a would need to be infinite. Is it because a would need to compensate for all the different values of $x_2$ regardless of $b$ being any arbitrary value? $\endgroup$ Dec 21, 2020 at 10:20
  • 1
    $\begingroup$ @AndréYuhai $a$ is the slope of the line. It gives you the change in $x_2$ for a unit change in $x_1$. For $a=1$, if you change $x_1$ by 1, $x_2$ will also change by 1. That's a 45° line. As $a$ grows, the line becomes steeper. For $a=1000$, increasing $x_1$ by 1 causes an increase of $x_2$ by 1000. That's already much steeper, but still not exactly vertical. Only when $a = \infty$, the line becomes vertical. It's easy to understand if you draw it up. $\endgroup$
    – Igor F.
    Dec 22, 2020 at 9:54
  • $\begingroup$ Oh, now I got it! Thank you! $\endgroup$ Dec 22, 2020 at 13:08

The equation is parametrizing a hyperplane.

You could set your decision boundary to be any parametrized by say Wp + b' = 1 if you wanted to. However, the results would be the same. Since this would be equal to saying the decision boundary is Wp + b'-1 = 0, and then we can define b=b'-1 and estimate b instead of b'


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.