Finding weights of a neural network by hand Given the four points $(x_1,x_2) = (0,0),(1,0),(0,1),(1,1)$, corresponding class labels $(1,1,1,-1)$ and activation units $y = f_H (w_0+\sum_i x_i w_i )$ and
$f_H(\alpha)$
$\begin{cases}
-1   ; \alpha < 0 \\
1  ;  \alpha \geq 0
 \end{cases}$
We have to show by specifying parameters that the above dataset can be classified with a single activation unit.
Plotting the points, I can see that (this NAND function) is linearly separable - but i) how do I show analytically that the solution only requires one activation unit? 
Furthermore, the four equations for $w_0,w_1,w_2$ seem to be underdetermined ii) what would be the systematic approach by hand (I can find solutions just by random guessing, but...)? 
Thank you
 A: This seems like a class/homework question (please add the self-study tag if that's the case), so I'll give a hint.
It looks like you're using the term 'activation unit' to mean output unit. Any binary classification problem can be solved with a single binary output unit; having only one of them isn't a restriction on the network. Rather, the restriction comes from the network being single layer, with a linear activation function. This means the network will only be able to achieve perfect accuracy on problems that are linearly separable.
As you've noticed, this problem is indeed linearly separable. There are uncountably many choices of parameters that will correctly classify your data points. But, to show existence, all you have to do is write down one of them. One strategy is to think about the problem geometrically. The decision boundary is a hyperplane because the network is a linear classifier; the output simply says which side of the hyperplane the input falls on. From your equations, we can see that the class is 1 if and only if $\vec{x} \cdot \vec{w} + w_0 \ge 0$. The decision boundary is $\vec{x} \cdot \vec{w} + w_0 = 0$. This is the equation for a hyperplane. The weights point in the direction of the hyperplane's normal vector. Adjusting the bias term shifts the hyperplane back and forth along this direction. The input is 2d, so it's easy to draw things out. Draw some line (i.e. 1d hyperplane) that separates the points. The equation for the decision boundary says how the line corresponds to network parameters.
