How to draw the single perceptron decision boundary when weights and bias are 0?

Question

I've been following an algorithm described on a book called Knowledge Discovery with Support Vector Machines by Lutz H. Hamel. In the book, there is this learning algorithm for a single perceptron below.

$$ \begin{align*} &\textbf{let} \quad D = \{(\bar{x_1}, y_1), (\bar{x_2}, y_2), ...,(\bar{x_l}, y_l)\} \subset \mathbb{R^n} x \{+1, -1\} \\ &\textbf{let} \quad 0 < \eta < 1 \\ &\bar{w} \leftarrow \bar{0}\\ &b \leftarrow 0 \\ &r \leftarrow max\{|\bar{x}|\, |\, (\bar{x}, y) \in D\ \}\\ &\textbf{repeat}\\ &\quad\textbf{for} \,i = 1\, \textbf{to}\, l\,\\ &\quad\quad \textbf{if}\, sgn(\bar{w}\cdot\bar{x_i} -b) \neq y_i\, \textbf{then}\\ &\quad\quad\quad\bar{w}\leftarrow\bar{w} +\eta y_i \bar{x_i}\\ &\quad\quad\quad b \leftarrow b - \eta y_i r^2\\ &\quad\quad\textbf{endif}\\ &\quad\textbf{endfor}\\ &\textbf{until}\, sgn(\bar{w}\cdot\bar{x_j}-b) = y_j \, with j=1, 2, ...,l\\ &\textbf{return}\, (\bar{w}, b) \end{align*} $$

So our decision boundary will be the line $\bar{w}\cdot\bar{x}=b$

Which can be written as,

$$ w_1 x_1 + w_2 x_2 - b = 0\quad\quad (1)\\ x_2 = -\frac{w_1}{w_2} x_1 + \frac{b}{w_2}\quad\quad(2) $$

---

Everything is fine until here and I want to animate the training of the perceptron using manim. Which will look something like this:

What I don't understand is, let's say I want to start my training with

$$ w = \begin{bmatrix} 0\\ 0 \end{bmatrix}\\ b = 0 $$

in this case, how can I draw the decision boundary? Because if we just substitute the values in equation (2) we get:

$$ x_2 = -\frac{0}{0}x_1 + \frac{b}{0} $$

• What's my decision boundary in this case?
• I've got another question about the GIF above. Should I draw the position vectors of the samples and the weight relative to the decision boundary and not the origin?

Answer 1

TL;DR: Your decision boundary is the whole $(x_1, x_2)$ plane.

In detail: The function

$$ z = w_1 x_1 + w_2 x_2 - b $$

is a plane in the 3D space, spanned by the axes $(x_1, x_2, z)$. Where $z = 0$, the plane intersects the horizontal plane, spanned by $(x_1, x_2)$. The intersection of these two planes in most cases happens to be a straight line, which we, in this context, call the 'class boundary'.

But, in general, the class boundary is the set of points where $z = 0$. In your special case, with $w_1 = w_2 = b = 0$, this is everywhere, regardless of the values $x_1, x_2$. So, the class boundary is the whole $(x_1, x_2)$ plane.

P.S. I'm impressed by your graphics.