I've read about pretrained word embeddings, and I understand how to use them. Basically, if I have the word nail (for example), there is a look up table where I can use the embedding for that word.

If we are now talking about pretrained contextual word embeddings, I understand that the embedding has information on the context around the word (either only words on the left, or both left and right).

So, sentences like "I feel I'm going to nail this task" and "I need a nail to hang up a picture", would produce different embeddings for the word "nail", am I correct?

However, I don't understand how these embeddings are then stored or reused. Is there a look up table with multiple entries for each word? If so, then how do we choose one?

Any help in understanding this is greatly appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.