For example, if the data is an acceleration, each input is the acceleration at a time T ?

enter image description here

More precisely, I'm curious if I can have the same behavior of a recurrent neural network but only with a normal artificial neural network which has "inputs over time".

In fact, I tried something like that and it doesn't work on my problem (but a RNN works). So I wonder if there is a mathematical explanation of why is it limited, why it may never have the same results?

  • $\begingroup$ Check out reservoir computing, such as liquid state machine and echo state network. Sounds similar to what you say. $\endgroup$ – Kolya Ivankov Nov 9 '17 at 9:38
  • $\begingroup$ Could you edit to tell us what do you mean by "imputing the same data over time"? Your description is very vague and makes it hard to answer. $\endgroup$ – Tim Nov 9 '17 at 9:41
  • $\begingroup$ Hi @Tim, I added a schema and, I hope, a sufficiently detailed description $\endgroup$ – dan32 Nov 9 '17 at 9:52
  • $\begingroup$ @dan32 so you mean applying NNs to something like time-series data? $\endgroup$ – Tim Nov 9 '17 at 9:54
  • $\begingroup$ Yes! If I understand correctly, in the case of a RNN we just input our data at the current time, and we modify the inside memory with this input when it learns. So I guess that a RNN has a memory over time, depending on the before inputs. But if I just need a short term memory (which is the case for a RNN, no?), can I just input my data at the current time, but also the 50 past values of my input? $\endgroup$ – dan32 Nov 9 '17 at 10:00

What you're suggesting is similar to regression approach to time series. For instance, you can recast the AR(1) process as a simple regression: $$y_t=\beta_0+\phi_1y_{t-1}+\varepsilon_t$$ will become OLS problem: $$y_i=X_i\beta+\varepsilon_i$$ where the independent variable $y_i\equiv y_t$and design matrix has two columns for intercept and bona fide one regressor $x_{1i}=1,x_{2i}=y_{i-1}$

This can be made work under certain conditions. Assumptions of OLS are slightly different (relaxed), of course.

Similarly, you could build the NN with one input which is the lagged dependent variable, which outputs the one step forecast. You'd have to call it repeatedly to produce the multi-step forecast. In this case your NN would basically degenerate into the glorified nonlinear regression $y_t=f(y_{t-1})+\varepsilon_t$

  • $\begingroup$ Thank you! But I'm afraid that I don't really understand the maths you wrote, I don't get the link between all the maths behind a NN and the equations you wrote... Can you enlighten me ? $\endgroup$ – dan32 Nov 9 '17 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.