All Questions
3 questions
2
votes
1
answer
433
views
How does Word2Vec CBOW softmax work with multiple context words?
I'm referring to following paper from Xin Rong - "word2vec Parameter Learning Explained", to be precise the equation (4):
$$
p(w_j|w_I) = \frac{\exp(\mathbf{v’}^{T}_{w_{j}}\mathbf{v}_{w_{I}})...
- 123
3
votes
1
answer
1k
views
Does hierarchical softmax of skip gram and CBOW only update output vectors on the path from the root to the actual output word?
After reading word2vec Parameter Learning Explained by Xin Rong, I understand that in the hierarchical softmax model, there is no output vector representation for words, instead, each of the $V-1$ ...
- 620
16
votes
3
answers
11k
views
Why is hierarchical softmax better for infrequent words, while negative sampling is better for frequent words?
I wonder why hierarchical softmax is better for infrequent words, while negative sampling is better for frequent words, in word2vec's CBOW and skip-gram models. I have read the claim on https://code....
- 47.6k