0
$\begingroup$

The word2vec model saves its layer weights as embeddings. But do CBOW and skipgram both store the input layer weights?

I know they learn different embeddings for the words in the context and for the word as center word. CBOW features the context embeddings in the input layer and Skip-gram the center words.

I had a look at the C-Code and it looks like they are both storing the input layer syn0 as word vectors.

Am I right in my understanding?

Are the CBOW vectors therefore the context vectors and not the center word vectors as in skip-gram?

Why does CBOW not store the center word embeddings (output layer)?

Thank you

$\endgroup$
0
$\begingroup$

As far as I know, both input and output weight layers could be used as word embeddings, and the input one is used by convention.

One of the answers here refers to a paper in which both input and output weights are used (Section 2 in A Dual Embedding Space Model for Document Ranking).

$\endgroup$
  • $\begingroup$ Thank you! So in CBOW the embeddings are the ones from the context words by convention? Thats what I thought. $\endgroup$ – Felix Feb 9 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.