3
$\begingroup$

I was following a youtube lecture about the topic and came upon this diagram (40:48) of the skip gram model which shows different outputs for each context word although also stating that the context word matrix is the same for every context word. It was my understanding that the output for each context word would be the same probability distribution and the loss would be calculated differently, but different online sources that I read say different things. If the context word matrix is the same how are different outputs produced?

$\endgroup$
1
$\begingroup$

In the paper word2vec Parameter Learning Explained by X. Rong, it is mentioned that "Each output is computed using the same hidden→output matrix" and each of the context word output will be calculated with this same matrix so, your understanding is correct that each word should have same probability distribution and loss will be calculated differently depending on the 1-hot vector for each context word.

Screenshot from X. Rong paper

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.