Why is the softmax on the dot product of the word embedding is the probability of context given word?

Question

I was learning about the Word2Vec model, and the following equation was shown:

$\huge{p(o|c) = \frac{exp(u^T_ov_c)}{\sum_{w\in{V}}exp(u^T_wv_c)}}$

in words, the probability of the context word given the center words is equal to the right hand side, where:

$u^T_o$ is the word vector for the context word

$v_c$ is the word vector for the center word

$w\in{V}$ is any word in the dictionary $V$.

I get the intuition for some of it. by using softmax on $u^T_ov_c$, we get a probability distribution that highlights the higher values while still applies some probability to the lower values (as opposes to hard max).

what I don't get - is what is the proof that the probability distribution on the right hand side it the same probability distribution as the left hand side? in other words, what justifies the equality sign in the equation?

I could not find the answer in the original Word2Vec paper.

Answer 1

This question arises from a misunderstanding of what a statistical model is.

Word2vec is a model which means that it's using a statistical procedure to estimate the probabilities given the learning procedure and the data. The equality sign is valid because the RHS is the estimated probability and this quantity is estimated using the values on the LHS.

If you used a different model, or even used a word2vec model with a different data set, you would estimate different probabilities.