3
$\begingroup$

For the XOR problem, 2 decision boundaries are needed to solve it using 2 inputs neurons, 2 hidden neurons, 1 output neuron. From the book "Neural Network Design" on page 84, the author says that "each neuron in the network divides the input space into two regions."

In the xor network, there are 3 neurons excluding the inputs. Therefore 3 decision boundaries should be drawn. But most books show only 2 decision boundaries for the xor problem. Can someone explain why is it so?

enter image description here

$\endgroup$
2
  • $\begingroup$ "In the xor network, there are 3 neurons excluding the inputs. Therefore 3 decision boundaries should be drawn." Why do you think this is so? $\endgroup$
    – ilanman
    Jan 2, 2017 at 23:03
  • $\begingroup$ because each neuron divides the input space into two regions as said above. Normally linearly separable problems have no hidden layer, therefore only 1 output neuron and it divides the input space into 2. So for non linearly separable (XOR) we have 2 hiiden nodes and 1 output node. Thats why I think so $\endgroup$ Jan 2, 2017 at 23:23

1 Answer 1

3
$\begingroup$

But there are 3 decision regions:

enter image description here

For example:

  • The first neuron splits the upper left blue input from the rest
  • The second neuron splits the lower right blue input from the rest
  • The output neuron splits the result into red area or blue area

Each neuron splits the input into one of 2 classes. Refer to Chapter 11 of that book for more detail.

For those interested, below is python code used to generate the plot. Not using neural-networks, just plotting a fabricated decision boundary:

import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
import numpy as np

boundary1 = np.linspace(-.2,1.2,100)
plt.plot(boundary1,boundary1+0.4,c='black')
plt.plot(boundary1,boundary1-0.6,c='black')
plt.scatter(1,1,c='red',s=100)
plt.scatter(0,0,c='red',s=100)
plt.scatter(0,1,s=100)
plt.scatter(1,0,s=100)
plt.fill_between(x=boundary1,y1=boundary1+.4,y2=boundary1+3,alpha=.2,color='blue')
plt.fill_between(x=boundary1,y1=boundary1-.6,y2=boundary1-2,alpha=.2,color='blue')
plt.fill_between(x=boundary1,y1=boundary1+.4,y2=boundary1-0.6,alpha=.2,color='red')
plt.xlim(-.2,1.2)
plt.ylim(-.1,1.1)
plt.show()
$\endgroup$
5
  • $\begingroup$ I was thinking about the same for the output neuron (red and blue area) .But the red or blue area does not seem to represent a boundary for me. According to me, it represents 2 planes :S $\endgroup$ Jan 3, 2017 at 17:31
  • $\begingroup$ but per your quote "each neuron in the network divides the input space into two regions". So there are 3 decision regions no? $\endgroup$
    – ilanman
    Jan 3, 2017 at 17:33
  • $\begingroup$ well I could not complete my comment. I was saying that I thought that a boundary should be represented by a LINE. The red and blue part are areas or atleast planes. Maybe I did not understand the concept of boundary correctly. $\endgroup$ Jan 3, 2017 at 17:42
  • 1
    $\begingroup$ I think the difference is between the definition and usage of the words boundaries and regions. $\endgroup$
    – ilanman
    Jan 3, 2017 at 17:45
  • $\begingroup$ yes. 3 neurons lead to 3 regions. 1 boundary separate 2 regions. therefore 2 boundaries are required to separate the 3 regions. $\endgroup$ Jan 3, 2017 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.