In this paper by Bengio on page 7 he used the attractor network to analyse recurrent network. And in this paper it says that attractor network dominates the field of memory control. What's more recurrent network features its memory ability compared to previous ones. so I'm wondering what kind of role does it play in the field of recurrent neural network or neural network nowadays? I raise this question because 1) I have asked some experts who told me they had never met this before and 2) in the popular deeplearning textbook by Ian Goodfellow, Yoshua Bengio and etc, it is almost not mentioned (only a reference can be found).
I also wonder what does an LSTM have to do with Attractors. Look at the paper "An Empirical Exploration of Recurrent Network Architectures", Proceedings of the 32 nd International Conference on Machine Learning, Lille, France, 2015 by Rafal Jozefowicz, Wojciech Zaremba Ilya Sutskever.
In the 2nd paragraph of Section 2.1 they conclude that LSTMs are not "compatible" with attractor systems based on a result of Bengio:
Bengio et al. (1994) showed that any RNN that stores onebit of information with a stable attractor must necessarilyexhibit a vanishing gradient. As Theorem 4 of Bengio et al.(1994) does not make assumptions on the model architec-ture, it follows that an LSTM would also suffer from van-ishing gradients had it stored information using attractors.But since LSTMs do not suffer from vanishing gradients,they should not be compatible with attractor-based mem-ory systems.