I have been trying hard to understand the concept of sampling some contexts. The main thing is I am unable to digest the idea of sampling. For example in Mikolov's papers, the negative sampling expectation is formulated as :
logσ(⟨w,c⟩)+k⋅𝔼cN∼PD[logσ(−⟨w,cN⟩)]
I clearly understand the left term logσ(⟨w,c⟩)
, but I can't understand the very idea of sampling negative word-context pairs.
How sampling works in Word2vec? Can someone please make me understand NCE and negative sampling?
Upendra Kumar
- 383
- 1
- 3
- 5