0
$\begingroup$

When people talk about using an autoencoder for feature extraction before some network, in this case a deep network of LSTMs, are they referring to first training the autoencoder, and then passing the latent representation to the next network, or are they talking about passing the reconstruction? What are the pros and cons? Would passing the latent representation of the autoencoder to a deep LSTM aid in time series analysis?

$\endgroup$
1
$\begingroup$
  1. I would train the AE, and use the resulting Encoder to extract a reduced dimension form of the input to pass to the LSTM. There is research that suggests the use of an AE has resulted in improved performance for prediction. Note that this is just dimensionality reduction, a non-linear form, as opposed to PCA, a linear form. There is research that suggests PCA helps to address multicollinearity, which constrains the predictive accuracy. I would think the Encoder would have similar positive effects.

  2. The reconstruction is used for training the AE. Use the reconstruction and the original input to compute a loss, compute gradients, and then apply backpropagation. Most frameworks will handle this for you out of the box. I don't think the reconstruction should be passed to the LSTM.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.