Question about Good Turing Discounting I've got a question about Good Turing discounting.  I understand the how and the why, but I'm having trouble wrapping my head around it.  
Say we're discounting the probability of each n-gram that occurs less than 5 times, so for every n-gram we use GT-discounting if it occurs less than 5 times, and the probability mass given to unseen events is N_1 / N.  If the training set follows a Zipfian distribution and has a large number of n-grams occur once, N_1/N is going to be a fairly large number, and the discounted probability mass of the other n-grams plus the mass for the unseen events will be greater than one.  
Is this correct, or am I missing something?  What is the probability of a single unseen event?  It can't be the N_1/N, that's far too large of a probability for a single unseen event.
 A: The motivation of smoothing methods is to "shift" the probability mass in order to provide a higher probability to unseen events.  The total probability mass remains therefore the same (it must sum to one, as for any probability distribution), so that means that the probability of seen events must be slightly reduced.  
That's exactly what Good-Turing discounting does: if you try to sum up the probabilities for all possible values, you'll see it sums up exactly to one -- if the calculations are tricky, it might be easier to convince yourself of this on a more basic smoothing method such as Laplace smoothing.
The value $N_1/N$ is the total probability for the collection of all possible unseen events. Although it might seem like a big number at first, the total number of possible unseen n-grams is also very large (orders of magnitude larger than the number of seen n-grams), so the probability of a single unseen event is not very big, namely $\frac{N_1}{N_0 N}$, where $N_1$ is the number of n-grams that appear once, $N$ the total number of n-grams, and $N_0$ the number of unseen n-grams.  The $N_0$ value can be derived by calculating the number possible combinations under a particular vocabulary, and substracting the n-grams that were observed.
