Language modeling: why is adding up to 1 so important?

Question

In many natural language processing applications such as spelling correction, machine translation and speech recognition, we use language models. Language models are created usually by counting how often sequences of words (n-grams) occur in a large corpus and normalizing the counts to create a probability. To account for unseen n-grams, we use smoothing methods (see several listed here) which take some of the probability mass from the n-grams which are attested in the model and distribute this mass among lower order n-gram (shorter word sequences) backoff probabilities.

Many of the smoothing techniques become mathematically complex because of the constraint that the calculations must keep the distribution as a probability (must add up to 1).

What is the reason for this constraint? What is the advantage of using strict probabilities for prediction instead of scores of any other kind?

P.S. The reference corresponding to the link is [Stanley F. Chen and Joshua Goodman (1998), “An Empirical Study of Smoothing Techniques for Language Modeling"].

Answer 1

The main advantages of using strict probabilities are a) ease of interpretation of the numbers; and b) being able to use Bayes theorem and other probabilistic methods in subsequent analysis. In some situations though, it is unnecessary. For example if you just want to rank the results with no further analysis, then there's no need to normalise the scores.