I'd like to train a RNN to predict a time series (The data set is composed of sequences of cell cycle division times). Iv'e seen two models that seem appropriate. Presented here is my understanding for how they work.

In the first, each training sequence is of some length $n$ and the goal of the model is to predict the sequence $\delta$ time steps into the future. So the target for each sequence would be that same sequence shifted $\delta$ time steps into the future. Then at test time, the input is some seed sequence $x$, and the output is $x'$, the prediction of $x$ shifted $\delta$ time steps into the future. We can then feed $x'$ into the model and get $x''$, and so on, generating a prediction of $x$'s future for an arbitrary length.

In the second model, at training time the target of any time step $t$, call it $y^{<t>}$, is fed into the hidden state of the next time step $h^{<t+1>}$ and at test time, instead of passing the real target $y^{<t>}$. we pass the predicted output $\hat{y}^{<t>}$, again, enabling us to generate a future prediction of an arbitrary length.

The more I think about it, the more it seems that really, the second model is actually a special case of the first with $\delta=1$. Is this observation correct? If not, what is the difference and when should I use each one? Are there other RNN architectures that may be relevant?

Here is a sketch of how these models should look (unrolled) enter image description here

  • $\begingroup$ Inputs are sequences of real valued numbers. Training data is composed of a few hundred such sequences, of varying length. The goal is to predict the future development of a given seed sequence. $\endgroup$ – H.Rappeport Sep 17 '18 at 15:10
  • $\begingroup$ The dimension at each time point is 1, a single scalar. I thought RNNs were much better in deducing temporal relations in sequences? Also, the input sequences may be of varying length. $\endgroup$ – H.Rappeport Sep 17 '18 at 15:52
  • $\begingroup$ My advice is first try traditional time series forecasting. You need this to establish the baseline anyways. I doubt you'll be able to beat TS analysis with RNN. $\endgroup$ – Aksakal Sep 17 '18 at 16:52
  • $\begingroup$ @Aksakal . Could you elaborate on what you mean by conventional TS analysis? $\endgroup$ – H.Rappeport Sep 17 '18 at 18:22
  • $\begingroup$ ARIMA, state space analysis, Hodrick Prescott filters, exponential smoothing etc. Are you familiar with time series forecasting? If not then you should get familiar, in my opinion, because for what you describe these things have a better chance to succeed than RNN $\endgroup$ – Aksakal Sep 17 '18 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.