What's the input to the decoder part of a sequence to sequence autoencoder? I've seen certain examples of such an autoencoder (using LSTM's more often than not) but am still unclear.

  • For example, here in this often-cited paper by Dai & Le ('Semi Supervised Sequence Learning'), we have the following diagram:

    Dai & Le

    What's the input to the decoder portion of the autoencoder here? In this example it's 'W-X-Y-Z.' But in general, is it the same as the input to the encoder? Or is it using the output from the previous timestep/LSTM cell as input?

  • Similarly, in another popular paper by Srivastava et. al ('Unsupervised Learning of Video Representations using LSTMs'), they have the following diagram:

    Srivastava et al

    It seems they're using the reversed input from the encoder as input here. However, there's a section as follows:

    The decoder can be of two kinds – conditional or unconditioned. A conditional decoder receives the last generated output frame as input, i.e., the dotted input in Fig. 2 is present. An unconditioned decoder does not receive that input.

    In the unconditioned decoder, what input does the decoder receive?

Thank you! I'm new here, so please be kind :)


If I understand the Semi-supervised Sequence Learning paper correctly, the input to the decoder-portion of the autoencoder is the same as it would be for an ordinary language model. The authors state:

We also find that recurrent language models [2, 24] can be used as a pretraining method for LSTMs. This is equivalent to removing the encoder part of the sequence autoencoder in Figure 1

Thus, it should be the same input that is fed to the encoder. The difference, then, would be that the hidden state of the decoder is initialized with the last hidden state of the encoder.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.