2
$\begingroup$

Let's say I have the following constraints:

  1. The architecture is fixed (see image) (note that there are no biases)
  2. Activation function for the hidden layer is ReLU
  3. There's no activation function for the output layer (should just return the sum of the inputs it receive).

![enter image description here

I tried to implement this in pytorch and train to learn XOR function but I failed.

I know that a single perceptron cannot learn the xor function because it always gives a linear boundary. Here, we are summing up two perceptrons which I believe that should be able to learn the XOR. Is this understanding correct?

My other questions are:

  1. Is this a feasible problem to achieve with the given network? If yes, how?
  2. If this is doable, can we still achieve that by constraining the weights to be in the set $\{-1, 0, 1\}$
|cite|improve this question
$\endgroup$
  • $\begingroup$ @asdf Can you prove your claim? $\endgroup$ – j.Doe Mar 7 at 20:43
  • $\begingroup$ Sorry, my previous solution does not actually work. I'm gonna think about it later, but it looks bad at first glance, since output(T,T) = output(T,F) + output(F,T) except for the sigmoid functions, which I don't really think as playing any role here $\endgroup$ – David Mar 8 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.