# Understanding word2vec backpropagation

I'm watching the following video on word2vec from University of Waterloo: https://www.youtube.com/watch?v=GMCwS7tS5ZM&t=962s

The update function for word my word embedding vector is:

$$v'_w = v_w - v_c(1 - P(w|c))$$, where

$$v'_w$$ = new word vector (embedding) of predicted word

$$v_w$$ = old word vector of predicted word

$$v_c$$ = old word vector of context word

$$P(w|c)$$ = "how well can I predict word given context"

The problem I'm having is that I don't see intuitively how this actually works. For example, if I initialize my $$v$$s to zero, I'll never actually make progress towards an accurate set of word embeddings, because the vectors will never move away from zero.

# Question

What part of this am I missing? I feel like I'm not seeing the entire picture.

I think here intuition is easier then derivation. If the context word, $$c$$ can describe well target word, $$w$$, then $$P(w|c)$$ will be closer to $$1$$ compared to $$0$$. And, we don't subtract the context from our word. If it is close to $$0$$, i.e. the context word cannot describe the target very well, subtract its context value from the current target. Initialization is another issue. A good way is initializing your weights to small, zero mean random values. Don't ever initialize all to zero. And, specifically in your case, it's a local minimum.