How to sample a language model?

Question

I've successfully trained a language model using LSTMs. But I have a confusion about sampling.

On sampling, we generate a probability distribution at each time step. It will be of length vocabulary size.

Now how to select a word from it?

Take the highest probable word (maximum probability) in this distribution and pass it as the next input? On implementation, I found it a cyclic prediction nature using this approach.

Or, say select top 5-10 most probable words and select one from them?

Or, randomly select any one word from the distribution and pass that one? In this case, what is the point of language modeling at all?

How to sample a language model?

Answer 1

Sampling a language model is a good way of understanding what the model has learnt.

You're right there's no point randomly sampling(with equal probability of each word at each time step t), then why have you even trained your model!?

We rather sample based on the probability distribution of $\hat{y^{t}}$. Eg: say probability of $\hat{y_{i}^{t}}=0.16$ (where $i^{th}$ index in probability distribution of our prediction for time step t) means that we select word at $i^{th}$ index with the probability 0.16. You do that in numpy using np.random.choice.

Also if you select the word with the highest probability you'll end up with the same sentence given the same initial word.

This is a great link to understand novel sampling.