I've seen a douzen of these question and I still feel like I'm not really grasping the concept. I've provided a picture that explains some of my concerns:
The picture to the right is acting like an AND-gate, with 3 lines. I keep wondering, is the three lines related to the fact that I have 3 hidden neurons, where the number of hidden nodes decides the number of discernible areas that I can have, but if this interpretation is correct, when would I want to use more hidden layers, what is the point? As a second question, let's presume that I have 3 hidden layers, and I have an activation function in the form of a sigmoid. My question is, is there any area that is too complex that a sigmoid function wouldn't be able to classify it for us? Because it seems to me that you are calculating the weighted sum, and then passing it through the activation function for each layer. As a last question, how does the sigmoid function really work for us, let's say I have like 1 hidden layer, using a sigmoid activation function with 10 hidden neurons, what is really happening, is the sigmoid function giving us building blocks in the form of sigmoid functions that I "build" with? I realize it was a lot of questions but this is really mindboogling me...