This is a simple question that is hard for me:

Let's consider simple sentence


and create training examples for skip-gram training (x, y) with number of skips= 2 and skip window = +1/-1:

(A, B), (A, D), (B, C), (B, A), (C, D), (C, B), (D, A), (D, C)

For continuous bag of words (CBOW) with same window parameters we will have same training examples set but in a different order:

(D, A), (B, A), (A, B), (C, B), (B, C), (D, C), (A, D), (C, D)

from machine learning perspective nothing changed and when we change window parameters but keep them same for both models training sets will change and will be the same for both models but with different order of training examples. Since training examples are same so vectors for neural network are the same, so it would seem that these models are the same. Where is the difference, what I am missing here?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.