Skip to main content
1 of 2
Upendra Kumar
  • 383
  • 1
  • 3
  • 5

How sampling works in Word2vec? Can someone please make me understand NCE and negative sampling?

I have been trying hard to understand the concept of sampling some contexts. The main thing is I am unable to digest the idea of sampling. For example in Mikolov's papers, the negative sampling expectation is formulated as : logσ(⟨w,c⟩)+k⋅𝔼cN∼PD[logσ(−⟨w,cN⟩)] I clearly understand the left term logσ(⟨w,c⟩), but I can't understand the very idea of sampling negative word-context pairs.

Upendra Kumar
  • 383
  • 1
  • 3
  • 5