Let's say I have the following constraints:
- The architecture is fixed (see image) (note that there are no biases)
- Activation function for the hidden layer is ReLU
- There's no activation function for the output layer (should just return the sum of the inputs it receive).
I tried to implement this in pytorch and train to learn XOR function but I failed.
I know that a single perceptron cannot learn the xor function because it always gives a linear boundary. Here, we are summing up two perceptrons which I believe that should be able to learn the XOR. Is this understanding correct?
My other questions are:
- Is this a feasible problem to achieve with the given network? If yes, how?
- If this is doable, can we still achieve that by constraining the weights to be in the set $\{-1, 0, 1\}$