1
$\begingroup$

A 3-layer network has two layers of connections (between input and hidden layers and between hidden and output layer). Doesn't this mean that the gradient "vanishes", at least slighty, when training the connections between input and hidden layer by backpropagation, because the connections between hidden and output layer have already been adapted?

$\endgroup$
1
$\begingroup$

No, the derivative of error with respect to connection weights is taken for the weights between input and hidden, and the weights between hidden and output. Thus two sets of derivatives are updated during each "sweep" involving one feed forward learning step and a backward propagation of gradient updates ("back-propagation"). Also, each layer has its own contribution to error, so while the I-H connection could have high error, the H-O side could have low error, or vice versa. For your problem, would recommend use of terminology "feed-forward single hidden-layer ANN with back-propagation" instead of three layer ANN.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.