Backpropagation: Is there a general weight update rule for both output and hidden layers? I'm looking for a general weight update rule for both hidden and output layers, no matter the number of layers, the connections or the transfer function. Does anything like this exist?
I'm quite new to backpropagation and neural networks in general, so if someone has an answer, I would be really grateful if it didn't include too many assumptions about my knowledge in the field.
 A: Here is a nice introduction for you to follow: http://www.mth.kcl.ac.uk/courses-09-10/guide.pdf
A: None of those factors should change the way you update the weights. All layers are generally updated in the same way.
Here is a basic idea of how to update the weights -
Back propogate the network and find the gradient of each weight with respect to a loss function. This gradient tells you how to change each weight in order to decrease the loss.
Multiply each of these gradients by a small constant (usually called the learning rate), 0.01 for example. This is done because updating each weight by the full gradient is too much and will cause it to explode.
Subtract each new scaled down gradient from each weight.
A: I copy from here: https://en.wikipedia.org/wiki/Backpropagation
but maybe there is some confusion?
It says, "backpropagation is a generalization of the (single layer) delta rule, made possible by using the chain rule to iteratively compute gradients for each layer." 
It is for multilayer(two or more) feed forward networks. If you look into it and try to understand how it works, you can alter it, for example by changing constants in the equation or adding terms that you find sound.
I hope that helps.
