All Questions
2 questions
1
vote
0
answers
173
views
When training word2vec, why is a new negative sampling process formulated instead of simply downsampling?
(For background, see The Skip-Gram Model.1 This question does not exactly use their notation, but you should be able to follow along.)
The original skip-gram log-likelihood of a single word, $w$, ...
- 1,625
7
votes
1
answer
2k
views
Do we use maximum likelihood or cross entropy Loss for training skip-gram model?
In the skip gram model, maximising the likelihood of the context words given the middle word is equivalent to minimising the objective function $J(\theta)$, where
$$J(\theta) = -\frac{1}{T}\sum_{t=1}^...
- 1,116