Output from Word2Vec

Question

I am working on sentiment analysis. I am using Word2Vec method. I don't understand the output from this code line.

 dum_out =  Word2Vec(size = 200,min_count = 10, window = 5 )

It gives a vector of 200 dimensions for each word(let us take a word "break"). So dum_out['break'] gives me a vector of 200 dimensions. I I understand that these numbers in the vector are probabilities of how close 'break' is to 'other words' . But my question is what are these 'other words' ? With what context are these figures calculated ? How does the 'dimensions' parameter affect the model ? (You can consider any vocabulary size if needed and any other parameters accordingly)

Answer 1

The numbers inside the vector are not probabilities. Indeed, they are not bounded between 0 and 1. Each number is just a real number, and the whole vector represents the embedding for a given word. To find the similarity between two words you are supposed to compute the cosine similarity (again, not a probability).

The way those numbers are computed depends on the implementation: common algorithms are SkipGram and CBOW. Basically a single layer neural network is trained to predict a word given some context (the size of the context is controlled by the window parameter) words (or vice versa), and after some epochs the layer provides word embeddings, that is vector representations of the words.

If you increase the value of the size parameter, you obtain a richer embedding for each word.

However, it looks like you are trying to do something without having a clue about what you are really doing. There is plenty of articles explaining word2vec, take a look at one of them: https://skymind.ai/wiki/word2vec