I'm programming a LSMT network from scratch (I'm interested in understandind their internal functioning), but my question applies to any RNN. There is somehting which I'm not sure to be doing right. Let's say I'm training my network with sequences $s_i$, each of length $l_i$ and with label $c_i$. This is what I'm doing, for each $s_i$:
- Reset the network internal state.
- Feed each element $s_i(1)$, $s_i(2)$, ..., $s_i(l_i-1)$, completely disregaring the output of the network.
- When the last element of the sequence, i.e. $s_i(l_i)$, comes along, I compare the output of the network with the class of the sequence, and apply BPTT.
Is this approach correct? It does make sense to me, but when I try to apply it to a slightly different situation I get confused.
Let's say that now I'm doing speech processing. Each sequence $s_i$ now comprises $n_i$ phonemes. For $n_i = 1$ I could of course apply the previous approach. But for greater numbers I'm not sure what I should do.