# How to sample a language model?

I've successfully trained a language model using LSTMs. But I have a confusion about sampling.

On sampling, we generate a probability distribution at each time step. It will be of length vocabulary size.

Now how to select a word from it?

Take the highest probable word (maximum probability) in this distribution and pass it as the next input? On implementation, I found it a cyclic prediction nature using this approach.

Or, say select top 5-10 most probable words and select one from them?

Or, randomly select any one word from the distribution and pass that one? In this case, what is the point of language modeling at all?

How to sample a language model?

• It's the third option. But what do you mean by "what is the point"? – shimao Aug 24 '18 at 2:41
• My understanding is that we get a distribution with maximum probability to minimum probability among the vocabulary. From this distribution, if we sample randomly, at the end of the day, what is really happening is a random selection of word. We don't really care about the maximum probable word. – Sleeba Paul Aug 24 '18 at 16:41

We rather sample based on the probability distribution of $$\hat{y^{t}}$$. Eg: say probability of $$\hat{y_{i}^{t}}=0.16$$ (where $$i^{th}$$ index in probability distribution of our prediction for time step t) means that we select word at $$i^{th}$$ index with the probability 0.16. You do that in numpy using np.random.choice.