2
$\begingroup$

I am learning LSTM implementation in torch from this code,it has these two lines of code:

-- transfer final state to initial state (BPTT)
initstate_c:copy(lstm_c[#lstm_c])
initstate_h:copy(lstm_h[#lstm_h])

I don't understand why we should put the last cell state and the last hidden state to the initial state? Could someone tell me why?

$\endgroup$

1 Answer 1

2
$\begingroup$

Typically this would be done when processing a long sequence as a series of smaller, contiguous blocks. The hidden state at the end of one block is used as the hidden state at the beginning of the next block. This is done because the blocks are contiguous in time, so the hidden state should carry over. Neglecting to do this would cause the network to forget its previous state.

$\endgroup$
1
  • $\begingroup$ Thanks, I finally understand. Also in the above code, it says dloss/dfinal_state is 0, do you know why this is true? $\endgroup$ Jul 12, 2016 at 7:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.