3
$\begingroup$

When learning neural networks one can often hear that Hinton proposed backpropagation in 1986.

After this big leap forward, we could train neural network efficiently.

But I have a question: How did we train a neural network without backpropagation before 1986?

Improve this question
$\endgroup$
2
  • $\begingroup$ I'm not saying you should always check Wikipedia, but you should check Wikipedia before you ask questions so you can gather more info $\endgroup$
    – Thomas Wagenaar
    Jul 1, 2017 at 19:16
  • 2
    $\begingroup$ This may be of interest. $\endgroup$
    – GeoMatt22
    Jul 2, 2017 at 2:09

1 Answer 1

Reset to default
1
$\begingroup$

I'm not very sure as to how much people used neural networks back then, due to limitations in computational resources. There are, however, other methods with which one can train neural nets!

  • Conjugate gradient, first proposed in 1952
  • Newton's method (i did not find any reference as to when the formula we use today in NN's was published)
  • Quasi - Newton method (again, I found no reference to original publish date)
  • Levenberg-Marquardt algorithm, first published in 1944

For more information on each method, refer to the following link : https://www.neuraldesigner.com/blog/5_algorithms_to_train_a_neural_network

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ Note that backprop is not an optimizer, but an algorithm to compute gradients. All the approaches you mention require gradients. $\endgroup$
    – GeoMatt22
    Jul 2, 2017 at 2:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.