I have some general understanding of how LSTMs Neural Networks function, but no experience. Does the following make sense in the context of how an LSTM network functions?

Imagine this process (loosely based on encryption):

Input Data -> Process A -> Process B -> Process C -> Output B -> Process D -> Output Data
                                \_ Output A _______________________/

A Process can be considered to be a mathematical or bitwise operation. Outputs are numerical data.

  1. Can LSTM memory units be used to store Output A and Output B per training step?
  2. Can data in the memory units be used within the same training step, for, say, some fully connected layers later on?

1) Assuming, you are looking to predict the next output value $Output_{t+1}$ given an input vector $[OutputA, OutputB]_{t}$, LSTM is not literally going to store the input vector for each time step. It will rather decide whether the input affects the next outputs. If yes, it will use this input vector and update its current state.

2) Implementations of RNNs include rolling it out upto certain time steps. So, yes, it will include(and/or exclude) the effect of not only current input ($t$) but also the inputs of past $t_{i}s$ upto the point you have unrolled your RNN.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.