2
$\begingroup$

I'm doing some research into neural networks and it seems like every single one i've come across implements a back-propagation algorithm. Is this because they're very easy to implement or are there other reasons?

Also, are there any papers comparing different neural network implementations?

Thank you.

$\endgroup$
1
$\begingroup$

Backpropagation, essentially, is just an algorithm to compute the gradient of a neural network using chain rule. Please refer to this question for details.

If your loss (error) function is differentiable with respect to your model's parameters, there's no reason not to use the gradient, since it tells at each point in the parameter space where to go in order to minimize the loss.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.