1
$\begingroup$

This question raised to me since I'm unsucessfully training a CNN-LSTM network atm. If for instance, LSTM requires a different type of BP algorithm (TBPTT), how do softwares deal with it? What is the correct way of handling this? Doing the same type of BP everywhere? Or changing in the middle of the architecture when reaching a different type of layer?

$\endgroup$
1
$\begingroup$

TBPTT the same thing as normal backpropagation. People rename "BP for RNNs" as "BPTT" in order to explicitly point out that the graph must be unrolled. But that doesn't change the algorithm itself, it's just a teaching aid. The "truncation" in TBPTT simply falls out as a logical consequence of the fact that you can only work on sequence of a finite length on physical hardware.

So there is no need to "switch" between them or anything like that, because they're the same thing!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.