Sorry for the beginner level question, but I am fairly new to the NLP world and am trying to better understand how word2vec is able to create useful word embeddings.
I'm looking for an intuitive explanation as to why, when creating word embeddings with a word2vec algorithm, the weight matrix should contain useful information relating to the word embeddings. As I understand it, when using either a CBOW or skip-gram model, we input one-hot vectors into a shallow neural network, and train the network to predict either the missing word, or the context words (depending on whether we're using a CBOW or skip-gram approach). Then we take the columns of the learned weight matrix to be the vector representations of the words in our vocabulary.
What I can't understand/internalize is exactly why the weight matrix generated by the network should actually have columns that represent useful word embeddings of each word (i.e. why should the network necessarily contain columns such that words with similar meanings are near each other in the embedding space?). It seems like you could imagine the network learning to create a weight matrix that does the specific task (predicting the missing word/predicting the context words) without the matrix having anything to do with word embeddings.
Perhaps I'm just missing something silly here, but if somebody could give an explanation to this, it would help me a lot!