The skip-gram model tends to predict the surrounding words or in other words, it tries to maximize the co-occurrence in the output of the Network. According to my knowledge, this makes a similar word closer in terms of cosine similarity. I am not able to understand how training the Neural network to predict surrounding words leads to greater cosine similarity between similar words. We are really taking the dot product between weights from the first layer( which are the word embeddings) and weights from the second layer in the output and not the dot product between word embeddings while training.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.