1
$\begingroup$

In neural language modeling, the target for each time step x(t) is the probability of x(t+1). But for sequence-to-sequence model, what's the target at each time step? Or, there is no target until the sentence end? If so, how to propagate error through time? It just propagate error once? I'm could't find the answer from papers since most of them don't introduce these details. Thanks a lot!

$\endgroup$

1 Answer 1

1
$\begingroup$

Sequence to sequence models consist of an encoder and a decoder. The encoder takes the input and has no target. The decoder is a language model that is conditioned on the output from the encoder. It's targets are the probability of x(t+1) just like a language model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.