How does using the same vector for the center word and for the context word impact the performance of word vectors in word2vec?

Question

By default, word2vec uses 2 vectors for each word: one for the center word and one for the context word:

$\color{steelblue}{\large \text{Word2vec: objective function}}$

$\color{darkred}{\scriptstyle{\bullet}} \quad \text{We want to minimize the objective function:}$

$$J(\theta) = -\frac{1}{T} \sum_{t=1}^T \sum_\underset{{j \, \neq \, 0}}{\scriptsize{-m \leq j \leq m}} \log P(\mathbb{w}_{t+j} \,| \, \mathbb{w}_t; \theta)$$

$\color{darkred}{\scriptstyle{\bullet}} \quad \underline{\text{Question:}} \; \color{orchid}{\text{How to calculate } P(\mathbb{w}_{t+j} \,| \, \mathbb{w}_t; \theta) \text{?}}$

$\color{darkred}{\scriptstyle{\bullet}} \quad \underline{\text{Answer:}} \; \text{We will }\textit{use two } \text{vectors per } \mathbb{w}\text{:}$

$\quad \color{steelblue}{\scriptstyle{\bullet}} \quad v_\mathbb{w} \, \text{when }\mathbb{w} \,\text{is a center word}$

$ \quad \color{steelblue}{\scriptstyle{\bullet}} \quad u_\mathbb{w} \, \text{when }\mathbb{w} \,\text{is a context word}$

$\color{darkred}{\scriptstyle{\bullet}} \quad \text{Then for a center word } c \text{ and a context word }o\text{:}$

$$\color{Orchid}{P(o\,|\,c) = \frac{\exp u_o^T v_c}{\sum_{\mathbb{w} \in V} \exp u_\mathbb{w}^T v_c}}$$

(source, mirror)

How does using the same vector for the center word and for the context word impact the performance of word vectors in word2vec?

Answer 1

See this post.

To summarize the post above, the goal of word2vec is to compute word embeddings such that semantically similar words are embedded closer to each other, and conversely, dissimilar words are embedded farther from each other.

In the neural network formulation, that means we minimize $$u_c^T v_c$$ because a word c is rarely in its own context. This minimization is not feasible if we constrain u to be same as v.