Confused on Neural Network Representation I'm reading about reppresenting neural networks with topology in this: http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/
And I don't understand the 5th image.  I understand how it makes a horizontal line with no hidden layer, but I don't get what exactly is happening in image 5.  
 A: I'm not an expert. This is pretty trippy, however, I think I have an idea:
In a multilayer neural network, each layer applies a transformation to the domain. One such transformation is by the tanh function, which is described in the section after figure 5. According to the weights and the offset, each point in figure four is shifted by the first layer applying the tanh transformation, which gives the figure five. Then the next layer, the output layer, which can do the same job as the neural network with no hidden layers ie. linear separation, can then linearly separate the the representation created by the hidden layer, as shown by the blue and red sections.
You can see the effect of this in the shape that is created. For values of x less than 0, tanh produces -1, and for values greater than 0, tanh produces 1, and you can see this pattern repeated in the representation in figure 5. If you read on, the material makes more sense, I think. For instance, figure six shows how the shape in figure five is achieved by the application of the transformation.
A lot of classification is about trying to linearly separate data.
I think the author could do a slightly better job of being explicit about what is being represented in each image and what the neural networks he is referring to look like.
