Based on the vector values of two words in word2vec, could we judge whether they co-occur and frequencies of coocurrence?


1 Answer 1


Let the two words be $a,b$ and let $u_a,v_b$ be the respective in/out representation vectors of each word (usually word2vec retains only the in representation, but both are needed for training). Then the probability of co-occurance within the training window size should be:

$$\frac{\exp(u_a\cdot v_b)}{\sum_i \exp(u_a\cdot v_i)}$$,

Where the sum is over your vocabulary.

  • $\begingroup$ Thanks, but what I mean is two in representation of two distinct words, does the usage of vector similarity measure like consine similarity helps in determine the concurrence of two words? $\endgroup$
    – william007
    Jul 4, 2019 at 5:30
  • $\begingroup$ @william007: No. it doesn’t make sense to measure co-occurrence with the same representation. Words are similar if they are used in similar contexts, but that doesn’t imply they co-occur $\endgroup$
    – Alex R.
    Jul 4, 2019 at 6:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.