NLP - how do you randomly draw negative samples? From my understanding, negative sampling randomly samples K negative samples from a noise distribution, P(w). The noise distribution is basically the frequency distribution + some modification on words. Typically we choose K = 5 ~ 20 negative samples.
P(w) = Uw(w)^(3/4) / normalization_factor
And I've seen these two same equations that are represented in two different notations:

Three questions:


*

*What is the meaning of the blue box? What is the significance of j and i?

*The second equation does not seem to show anything that "randomly draws" words from the noise distribution. What is the meaning of k in red box?

*How do you choose noise samples from the noise distribution?


Let's say that the normalized noise distribution looks as the following dictionary:
dist = {'apple': 0.0023, 'bee': 0.004, 'desk':0.032', 'chair': 0.032 ...}
How do you "randomly draw" K noise samples from dist?
 A: 
What is the meaning of the blue box? What is the significance of $j$ and $i$?

The notation $\mathbb{E}_{j \ \sim \ P(w)}$ in the blue box would typically mean that you are taking the expectation of the quantity that follows, where $j$ is considered to be a random variable with distribution $P(w)$.  As to the index $i$ used in the summation, this part does not even make sense, since there is nothing in the summed quantity that explicitly uses the index $i$.  Presumably the authors have either made a mistake in their notation or one of the quantities in the sum is intended to depend on $i$ implicitly.
A: I can only speak to your third question. There may be a better way to do this but the simplest way to do this in python that I see is to split your dictionary into an array of keys and an array of values.
    import numpy as np
    MyDict = {'A':0.4, 'B':0.2, 'C':0.3, 'D':0.1}
    keys = np.asarray(list(MyDict.keys()))
    values = np.asarray(list(MyDict.values()))

Then I would call Numpy's random.choice method passing the array of values as the probabilities. For example:
    actual_samples = {'A':0, 'B':0, 'C':0, 'D':0}
    for i in range(1000):
        letter = np.random.choice(keys, p=values)  # gets one sample at a time
        actual_samples[letter] = actual_samples[letter] + 1

You can grab k samples at a time using the 'size' parameter. But to prove the point, the actual_samples dictionary above can be printed and you will see that the samples are coming from the intended probability distribution.
