What is the difference between recurrent neural networks and Hopfield networks, or are they same?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
They are not the same. A Hopfield network is one particular type of recurrent neural network.
Take a look at Chapters 14 and 15 of Haykin, Neural Networks. A recurrent neural network is any neural network in which neurons can be connected to other neurons so as to form one or more feedback loops (i.e. not like in a multilayer perceptron where everything goes one way - see the pictures in this question.) There are basically two useful kinds of recurrent network at the moment. One kind are those that try to simulate the human memory. The Hopfield network is a particular example. The other kind are input-output mapping networks, which can be used for calssification and prediction of time series data.
answered Aug 14, 2014 at 22:06
$\begingroup$
$\endgroup$
According to Wikipedia: "The Hopfield network is an RNN in which all connections are symmetric." Other types of RNN that are not Hopfield networks are: Fully reconnect, recursive, Elman, Jordan and more.
Further: RNN is a network were output can also be an input to the network. See this image to illustrate (taken from the video referenced below):
Hopfield network is just a recurrent network like this one, where the weight from node to another and from the later to the former are the same (symmetric).The Hopfield network is fully connected, so every neuron’s output is an input to all the other neurons. Another feature of the network is that updating of nodes happens in a binary way. These features allow for a particular feature of Hopfield's nets - they are guaranteed to converge to an attractor(stable state).
See: A friendly intro to RNN .
