How to find weights and threshold of decision boundaries?

Question

I have a question regarding multi-layer perceptron in neural network. So basically as can be seen from the image below, it is an input space where every pattern inside the triangle has target output O = 1 and patterns outside the triangle O = 0.

Given the following input patterns and their target outputs:

mu | x1 | x2 | O
1    0    -2   0
2    0     1   1
3    1     1   1
4   -3     4   0
5    4     4   0

we try to solve this classification problem by using a neural network with 2 input, 3 hidden neurons and one output.

How can I find the weights and thresholds of the decision boundaries? From my experience, weights are perpendicular with the decision boundary and thresholds are how much the decision boundaries shift from origin, But using these assumption, I keep getting the classification wrong for some patterns.

Thanks in advance

Answer 1

Suppose $f(x)$ is the real-valued output of the network for input $x$. For example, if the output unit is sigmoidal, then $f(x)$ represents the probability that the class is 1. To form a decision, the real-valued output is compared to a threshold $t$. The predicted class is 1 if $f(x) > t$, otherwise 0. The decision boundary is the solution to the equation $f(x) = t$.

For linear classifiers (e.g. typical neural nets with no hidden layer), the decision boundary is a hyperplane (i.e. line in your 2d example). But, your network has a hidden layer. If hidden units have a nonlinear activation function, the decision boundary will be nonlinear too. Its shape will depend on the details of your network and the parameters you've learned.