I've read a couple online description of CBOW and Skip-Gram and usually the descriptions starts like this:
- We need to train models on words
- So we encode words using vectors
- One-hot encoding is not efficient representation, we deal with this issue with CBOW and Skip-Gram
- CBOW is a model that allows you to predict the center word given the surrounding context
- Skip-gram does the reverse, by allowing you to predict the context given a center word
But wait. The problem we had (at step 3) was that one-hot encoding does a poor job at representing words. How does predicting a word help with the encoding process?! It seems we completely forgot about what we were setting out to do.