I have a 2-layer fully connected ReLU network with weights and biases $W_1 = [[-4, 1], [2, 3]], b_1 = [3, -2], W_2 = [[2, 3]], b_2 = -2$. The ReLU non-linearity is only between the layers, while the output is in the range $(-\infty, \infty)$.

Now, I would like to draw the decision boundary of this network. After applying all the weights, biases and the activation function on an input $(x_1, x_2)$, I end up with the following expression:

$2max(0, -4x_1+ x_2 + 3) + 3max(0, 2x_1 + 3x_2 -2) -2$

After setting the expression to zero, I try to come up with a way to plot the decision boundary in the $x_1, x_2$ plane, but I am stuck. Apart from trying to guess numbers $x_1 $ and $x_2$ that evaluate the expression to zero, is there a systematic way to plot the decision boundary?


1 Answer 1


There are four regions depending on the +/- status of the lines $-4x_1+x_2+3=0$ and $2x_1+3x_2-2$. You'll evaluate the expression for these four regions separately. For example, for the region where both lines give positive values, you'll end up with the boundary line: $$2(-4x_1+x_2+3)+3(2x_1+3x_2-2)-2=-2x_1+11x_2-2$$ Evaluate and draw the boundaries with these four regions.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.