0
$\begingroup$

I am trying to understand the advantage of a RNN has over a regular FF NN that is fed all the time series data.

It seems that advantage a RNN has over a feed forward network if you wanted to train the network after an initial training, with new datasets, if would be computationally much more efficient, since the state is saved. Where as you in a FF NN, you would need change the structure of the network itself, and train over all the time steps again.

But as for having one set of time-series data, I don't see how a RNN would have an advantage, since the neurons in a FF NN can learn relationships between the data points. My intuition doesn't how how the structure of a RNN would do this more efficiently.

I have searched questions and the closest thing I can find is this one Mathematical justification for using recurrent neural networks over feed-forward networks

Which gives some great thoughts related to the question I asked, but doesn't give an exact answer to my question.

$\endgroup$
  • $\begingroup$ “As for having one set of time series data”—can you clarify $\endgroup$ – user0 Jan 11 '18 at 1:38
  • $\begingroup$ Say that for example, you have data from data from 20 different time points, and you can train both a regular NN and you can train a RNN over that data. Now say that you have received data from 2 more time points, so now you have data from 22 different data points. For a regular NN you would need to built a new NN and train over the 22 data points, and for a RNN, you can keep the same structure, and use re-use calculations from when the RNN trained on the original 20 data points. $\endgroup$ – SantoshGupta7 Jan 11 '18 at 2:06
  • $\begingroup$ The loss might backprogagate over time differently now that you’ve added two time points, if I understand you correctly. The real advantage is that in the situation you’ve just described (assume the data is 1-dim), you have only 3 parameter (hidden to hidden, hidden to op and input to hidden)for the RNN and 22 or more for the NN. $\endgroup$ – user0 Jan 11 '18 at 3:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.