Can this network learn the XOR function?

Question

Let's say I have the following constraints:

1. The architecture is fixed (see image) (note that there are no biases)
2. Activation function for the hidden layer is ReLU
3. There's no activation function for the output layer (should just return the sum of the inputs it receive).

I tried to implement this in pytorch and train to learn XOR function but I failed.

I know that a single perceptron cannot learn the xor function because it always gives a linear boundary. Here, we are summing up two perceptrons which I believe that should be able to learn the XOR. Is this understanding correct?

My other questions are:

1. Is this a feasible problem to achieve with the given network? If yes, how?
2. If this is doable, can we still achieve that by constraining the weights to be in the set $\{-1, 0, 1\}$

Answer 1

Assuming your neurons have bias terms (call them $b_1$, $b_2$, and $b_3$), your solution is:

$$w_1 = 2, w_3 = -2, b_1 = -1$$

$$w_4 = -2, w_2 = 2, b_2 = -1$$

and

$$w_5 = 1, w_6 = 1, b_3 = 0$$

What is actually happening:

The first (top) hidden neuron "fires" (produces $1$ as the output) on input $[1, 0]$ and is inactive otherwise. Conversely, the other hidden neuron fires only on $[0, 1]$. In the output layer we sum these values.