5
$\begingroup$

So I've been watching lectures and doing the problem sheets from the class CS 229 2017, taught at Stanford by Andrew Ng. In problem sheet 3 he puts forward the following question:

enter image description here

enter image description here

$\textbf{So this is what I did in part a)}$

What we have is:

$\sigma(z) = \frac{1}{1+e^{-z}}$ and it's derivative is $\sigma'(z)=\sigma(z)(1-\sigma(z))$

$l=\frac{1}{n} \sum^{n}_{i=1} (o^{i}-y^{i})^2$

From this we can calculate:

enter image description here

$\textbf{Then for part b)}$ Here I'm completely lost. I'm guessing the triangle that he talks about is the following:

enter image description here

However, I don't understand how I would choose the weights in such a way that I would achieve 100% accuracy. Any help or hint would be very well appreciated!

$\endgroup$
15
$\begingroup$

There's three sides to the triangles and three hidden neurons. You want each hidden neurons to check on which side of the triangle side an input is. So:

  • The first hidden neuron will represent $x_1 > 0.5$
  • The second will represent $x_2 > 0.5$
  • The third will represent $x_1 + x_2 < 1$

Then the output will be something like a logical "and", or in other words, a sigmoid with an activation threshold.

$\endgroup$
  • $\begingroup$ A sigmoid with activation threshold separates the plan with straight lines? My intuition would tell me that the frontiers should be curved since the sigmoid is curved? Could you explain me where I am wrong? $\endgroup$ – Max Mar 29 '18 at 19:38
  • 2
    $\begingroup$ The straight lines aren't exactly caused by the sigmoid function. They come from the weights: an equation like ax_1 + bx_2 + c = 0 represents a straight line, and this weighted sum is exactly the input to the sigmoid functions. $\endgroup$ – Greg d'Eon Mar 29 '18 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.