0
$\begingroup$

I have high-dimensional events that I want to feed into an LSTM.

I was planning to pre-learn a context embedding (like word2vec), as opposed to learning an embedding as the first layer in the network, so that I can use the same embedding for multiple different prediction tasks, but I don't know if this is the best idea. If I end up doing it this way, is it best to separate out a portion of the training set to learn the embeddings, or can I learn the embeddings over the whole training set before re-using the training set to train the whole network?

I am leaning towards using the whole training set to learn the embeddings, as it feels analogous to the typical setup of adding a single embedding layer to the whole model, but I'm not confident in my assumptions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.