Getting started with dynamic neural networks

Question

I was wondering if anybody had some good references on dynamic neural networks. The goal is to have a neural network that responds in real-time (and is governed by say a system of DEs) instead of at discrete time-steps. The prototypical application is a robot: we have a series of sensors that are constantly feeding data into the input neurons, and we have output neurons that are constantly telling the motors what to do. We want to optimize this robot to do some tasks.

Is there a nice recent survey on dynamic neural nets? In particular one that discusses recurrent DNN, feedforward DNN, and DNN with and without learning.

Alternatively, if continuous dynamic neural networks are not the right approach for this sort of task, then suggestions of other types of neural nets are also welcome.

Notes

I also cross posted this question to MetaOptimize.

Answer 1

One way to make your NN dynamic is to use a dynamic training algorithm, i.e., an algorithm that would dynamically adapt the found solution. I did some research on this for my MSc thesis, and discovered dynamic particle swarm optimisation (PSO) to be rather efficient. Dynamic PSO showed results superior to that of backpropagation on a selection of dynamic classification problems, under a range of different dynamic scenarios. I found out that dPSO is more efficient than BP under gradually changing environments - i.e., PSO finds more precise solutions, and tracks them better than BP, provided the solutions change gradually (in the spatial sense). I'm currently working on a paper for the Springer Swarm Intelligence journal on the topic, but you can check out the thesis itself here (warning: it's lengthy! :P). A shorter, but rather limited version was presented at a conference (I consider this paper somewhat outdated). Good luck and keep us posted ;)