0
$\begingroup$

If the weights are trained with enough examples, then they shouldn't be 0.

From what I understand the sparsity problem of word2vec comes from the fact that most of its weights are 0.

Word2vec has weights from the input to the projection and from the projection to the output.

$\endgroup$
2
$\begingroup$

Be more precise about the "sparsity problem" of word2vec.

The weights learned by word2vec are the dense word representations of dimension d. These weights are not equal to 0.

The only sparsity I see is the encoding of the input words : they are encoded using "one-hot encoding". This means that when you have V words in your vocabulary, each word is encoded as a sparse vector of dimension V.

I suggest you to take a look at this site which explains word2vec.

http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model/

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.