I'm watching the following video on word2vec from University of Waterloo: https://www.youtube.com/watch?v=GMCwS7tS5ZM&t=962s
The update function for word my word embedding vector is:
$v'_w = v_w - v_c(1 - P(w|c))$, where
$v'_w$ = new word vector (embedding) of predicted word
$v_w$ = old word vector of predicted word
$v_c$ = old word vector of context word
$P(w|c)$ = "how well can I predict word given context"
The problem I'm having is that I don't see intuitively how this actually works. For example, if I initialize my $v$s to zero, I'll never actually make progress towards an accurate set of word embeddings, because the vectors will never move away from zero.
What part of this am I missing? I feel like I'm not seeing the entire picture.