I'm trying to understand how a skip-gram model trains itself. Using the input embedding we predict the probability of the output word. And then using the gradient of the cost function we slightly update embedding of the output. But also we update weights.

So I cannot understand how we simultaneously update weights and embeddings? Do they use different gradients or there is some sort of chaining?

I'm trying understand the logic of training weights and embeddings using this example (github).

  • $\begingroup$ Yes, I figured out that it is indeed updated in the backpropagation training. Thank you for a hint :) $\endgroup$ – Vadym B. May 17 '17 at 16:53