How to understand Neural network (with tensorflow)? [closed]

Question

I found this very awesome playground for visualizing the output of a neural network

I try to understand how the neural network works with basic dataset:

• for circle dataset, it is possible to build a nnet with 2 features ($X_1^2$ and $X_2^2$) and 0 hidden layer.
• for exclusive or dataset, we can use the $X_1X_2$ feature and 0 hidden layer
• for gaussian dataset, we can also use $X_1$ and $X_2$ features and 0 hidden layer

It is possible to obtain the "good" solutions with this above. Now when it comes to spiral dataset, I don't know how to find a simple solution.

How could you explain, with a particular dataset, the importance of hidden layers? For example, for dataset (except spiral dataset), what is your suggestion to mitigate the predictions and why?

Answer 1

The hidden layers are necessary for capturing interactions between features in the data. Without any hidden layers, the neural network is logistic regression (with sigmoid activation) or a perceptron (with linear activation).

The process of multiplying weights and inputs and then using an activation function on the output (and then further multiplying that output with different weights and using another activation function, etc.) which allow the ANN to model the non-linearities in the data.

(While the circle data is non-linear, the features $X^2_1$ and $X^2_2$ allow the logistic regression to classify the data correctly.)