Being recently interested in Kalman filters and Recurrent neural networks, it appears to me that the two are closely related, yet I can't find relevant enough litterature :
In a Kalman filter, the set of equations is : $$x_{k} = Ax_{k-1} + Bu_{k} + w_{k-1}$$ $$ z_k = Hx_k + v_k$$
with $x$ the state and $z$ the measurement.
In an Elman RNN (from here), the relation between the layers is: $$h_{k} = \sigma_h (Uh_{k-1} + Wx_{k} + b)$$ $$ y_k = \sigma_y (Vh_k + c)$$
with $x$ the input layer, $h$ the hidden layer and $y$ the output layer and $\sigma$ are the activation functions for the layers.
It's clear that the two set of equations are the same, modulo the activations. The analogy here seems to be the following. The output layer corresponds to the measured state, the hidden layer is the true state, driven by a process $x$ which is the input layer.
First question : is the analogy viable ? And how can we interpret the activations ?
Second question : in a Kalman filter the $A$ matrix is that of the underlying dynamics of the state $x$. Since training a RNN allows to learn the $W$ matrices, are RNN able to learn the dynamics of the underlying state ? Ie once my RNN is trained, can I look at the coefficients of my network to guess the dynamics behind my data ?
(I'm going to try to do the experiment on artificially generated data, to see if this works, and will update as soon as it's done)
EDIT : I wish I had access to this paper