I am a total novice to this, but my understanding is the following:
input layer - one neuron per input (feature), these are not typical neurons but simply pass the data through to the next layer
hidden layers - simplest structure is to have one neuron in the hidden layer, but deep networks have many neurons and many hidden layers.
output layer - this is the final hidden layer and should have as many neurons as there are outputs to the classification problem. For instance:
- regression - may have a single neuron
- binary classification - Single neuron with an activation function
- multi-class classification - Multiple neurons, one for each class, and a Softmax function to output the proper class based on the probabilities of the input belonging to each class.