# Plotting the decision boundary of a ReLU 2-layer network

I have a 2-layer fully connected ReLU network with weights and biases $$W_1 = [[-4, 1], [2, 3]], b_1 = [3, -2], W_2 = [[2, 3]], b_2 = -2$$. The ReLU non-linearity is only between the layers, while the output is in the range $$(-\infty, \infty)$$.

Now, I would like to draw the decision boundary of this network. After applying all the weights, biases and the activation function on an input $$(x_1, x_2)$$, I end up with the following expression:

$$2max(0, -4x_1+ x_2 + 3) + 3max(0, 2x_1 + 3x_2 -2) -2$$

After setting the expression to zero, I try to come up with a way to plot the decision boundary in the $$x_1, x_2$$ plane, but I am stuck. Apart from trying to guess numbers $$x_1$$ and $$x_2$$ that evaluate the expression to zero, is there a systematic way to plot the decision boundary?

There are four regions depending on the +/- status of the lines $$-4x_1+x_2+3=0$$ and $$2x_1+3x_2-2$$. You'll evaluate the expression for these four regions separately. For example, for the region where both lines give positive values, you'll end up with the boundary line: $$2(-4x_1+x_2+3)+3(2x_1+3x_2-2)-2=-2x_1+11x_2-2$$ Evaluate and draw the boundaries with these four regions.