All Questions
2 questions
0
votes
0
answers
40
views
How does the chain-rule look for the gradient of a loss function?
When we are computing the gradient of the loss function, $L$, of a Word2Vec model, for the context word-embedding, $w_i$, and the target word-embedding, $t$. Where the loss function, $L$, looks like:
$...
- 1
7
votes
1
answer
2k
views
Do we use maximum likelihood or cross entropy Loss for training skip-gram model?
In the skip gram model, maximising the likelihood of the context words given the middle word is equivalent to minimising the objective function $J(\theta)$, where
$$J(\theta) = -\frac{1}{T}\sum_{t=1}^...
- 1,116