0
$\begingroup$

In Hopfield networks, one can apparently load perfect recall into the network (by having enough neurons compared to patterns). (Source)

However, at the same time, it appears that spurious states (i.e. local minima of the energy function) will always exist. For example the opposite of a stored pattern is also a stable state.

These two statements seem incompatible to me, how can we have perfect recall despite the existence of unintended local minima?

$\endgroup$
1
$\begingroup$

By coincidence, I have been wondering the same thing and after some reading, I believe to have found the answer.

In "The capacity of the Hopfield associative memory." by McEliece et al. they describe the perfect recall of the trained points without error to be the ability of the network to recall all of the trained points. This means that the trained points have to be attractors of the network. This does not mean that these should be the only equilibrium points of the Hopfield network. Spurious states can not be avoided, but they do not prevent the network to recall the trained points.

$\endgroup$
  • $\begingroup$ Thanks Tom, but, unless I misunderstand you, this doesn't completely demystify to me why spurious states don't prevent the network to recall the trained points. I have been thinking about it a bit more as well, and wonder whether there is an implicit assumption about the amount of noise that is expected, eg, the variance of the noise won't exceed X such that we will always evolve to the trained points (?) $\endgroup$ – GR4 Jun 20 '16 at 15:08
  • $\begingroup$ Inputs to Hopfield networks are supposed to converge towards the equilibrium point closest to the input. According to the same paper I cited in my answer, to have perfect recall properties, the input has to converge to the nearest stored point. When the capacity of the network has been reached, this convergence becomes imperfect and the result may not be one of the memories. In other words, even if your input pattern is closer to a trained memory, the network may converge to a spurious pattern. I believe this is what is meant by perfect recall. $\endgroup$ – Tom Jun 20 '16 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.