# In an election, how can we tell the certainty that a candidate will be the winner?

There was a general election where I live yesterday and the television network started calling out winners long before all ballots were opened.

They turned out right on all accounts, and I'm not really surprised they did. I know that statistics are absolutely viable. Still, I'm curious. Assuming:

• we have opened $i$ out of $j$ ballots;
• we have $n$ candidates whose current scores are $c_1, c_2, c_3, ... c_n$;

How can we calculate the certainty with which the leading candidate is the winner?

• Bear in mind that they typically have access to extensive exit polling data, & other data that they can use to predict the result. They only need enough confirmation from the incoming count to ensure that they're not off the mark due to sampling error. There certainly are complexities involved & the incoming counts are generally a biased sample, but exit polls go a long way towards helping them address some of those issues. Sep 6, 2012 at 18:35
• If "with certainty" is meant to be taken literally, statistics can (almost?) never answer a question "with certainty". But we can give answers with a high level of confidence that the answer will be correct. (In other words, if we get our data and do our analyses correctly, we can say things like, "My answer will only be wrong about x% of the time.") Sep 11, 2012 at 19:10

The main difficulty in practice is not the statistical uncertainty that a fluke streak of luck would have given one candidate more votes. The main difficulty, by an order of magnitude or more, is that the ballots which have been opened are almost never an unbiased sample of the votes cast. If you ignore this effect, you get the famous error "Dewey Defeats Truman," which occurred with a large biased sample.

In practice, voters who favor one candidate versus another are not equally distributed by region, by whether they work during the day, or by whether they would be deployed overseas hence would vote by absentee ballots. These are not small differences.

I think what news organizations do now is to break the population into groups and use the results to estimate how each group voted (including turnout). These may be based on models and prior assumptions based on previous elections, not just the data from this election. These may not take into account oddities such as the butterfly ballots of Palm Beach.

• In Australia up until around 10-15 years ago, the conservative parties usually started strong in the early counting, with the progressive parties making a late comeback. The TV networks probably knew what was going on, but the variability probably made for more drama. It all changed when an analyst name Antony Green started using the booth by booth results to account for the fact that small booths in rural areas tend to get their counts done and results in early, and they tend to vote more conservatively. Antony famously correctly called an election result hours before anyone else using this. Sep 6, 2012 at 0:18
• Booth by booth results from previous years can be used to very accurately calibrate the estimates of the total result. Sep 6, 2012 at 0:27
• @DouglasZare I think you mean that the ballots currently opened is not a random sample. Sep 6, 2012 at 0:48
• @Michael Chernick: What is the difference between a nonrandom sample and a biased sample? en.wikipedia.org/wiki/Sampling_bias seems to use them as synonyms. Sep 6, 2012 at 1:00
• @DouglasZare I see from your link that wikipedia uses biased sample as a synonym for non-random. I think that is a poor choice. Bias general refers to the expectation of an estimator not being equal to the true valu of the parameter. In the context of sampling a non-random sample does not imply bias for a particular estimate. It may or may not lead to bias. Sep 6, 2012 at 1:05

In survey sampling the standard error of the estimate of proportion is needed. It depends more on i than j. Also it requires that the i opened ballots were selected at random. If p is the true final proportion for candidate A, then the variance of the estimate is

$$\frac{(1-\frac{i}{j})p(1-p)}{i}$$

The quantity $(1-\frac{i}{j})$ is called the finite population correction factor. To estimate this variance the usual estimate for p is substituted for p in the formula. The standard error is gotten by taking the square root. In predicting a winner the pollster might use the estimate plus or minus 3 standard errors. If 0.5 is not contained in the interval, then Candidate A is declared the winner if 0.5 is below the lower limit, or his opponent is declared the winner if 0.5 is above the upper limit. Of course this only says with very high confidence who the winner will be in the event that 0.5 is outside the interval. The confidence level is 0.99 if three standard errors is what you use (based on the normal approximation to the binomial). If 0.5 is inside the interval no one is declared the winner and the pollster waits for more data to accumulate.

In making a projection the pollsters can select a stratified random sample from the accumulated votes to avoid potential bias that mmay occur if one looks at all the counted ballots. The problem with looking at all accumulated votes is that certain precincts complete counting over others and they may not be representative of the population.

The article here provides good coverage of the problem and numerous references.

It has been pointed out that accumulated votes can provided biased estimates of proportions because either the precincts that have yet to report are precincts that tend to favor the party with the candidate that is trailing or the absentee ballots are likely to favor the candidate that is trailing and those votes get counted last. The sophisticated pollsters like Harris and Gallup do not fall into such traps. The simple analysis of constructing confidence intervals based on accumulated votes that I have outlined is only one factor that is used. These pollsters have a great deal more information at their disposal. They have polls that were taken shortly before the election and they have the voting patterns of all the precincts and absentee votes taken in elections in recent past years.

So if there are clear biases that could swing a close election in the opposite direction the pollsters will recognize this and hold off projecting a winner.

In the US absentee ballots come predominantly from the military overseas and college students who are at school away from home. While the military may tend to be more conservative and likely to vote Republican, the colleage students tend to be more liberal and likely to vote Democratic. All these considerations are taken into account.

The care and sophistication of modern polling is the reason that gross errors such as the Literary Digest poll of 1936 or the Chicago newspaper's premature concession of the 1948 election to Dewey have not occurred since then.

• Although the implicit analogy with survey sampling is apt, doesn't this question add complicating factors? First is the possibility of more than two candidates. Second is that this is a sequential decision problem: unlike the pollster, who typically specifies a poll size and makes one decision based on the sample, at each moment the network has a growing sample and must decide whether to call the election or to wait for more information. The survey applications you quote here do not seem applicable to this dynamic situation. And why would the network use 3 SE's? (Its reputation is at stake.)
– whuber
Sep 5, 2012 at 17:08
• @whuber I agree that there are complications which probably are not considered in practice. I chose for simplicity a two candidate case where majority is a win. I think this is the situation that the OP had in mind. Winning by plurality with three or more candidates would involve showing that the "winning candidate had a higher proportion than his opponents. Certainly if you do the poll more than once the sequential nature of the sampling should be taken into account. I am not sure that it is. Sep 5, 2012 at 17:18
• My choice of 3 SE was because I think the pollsters want to be "very sure" that they are right before declaring a winner. Hence I think 3 would be used over 2. If you want an even smaller risk of error you might go higher than 3. I used the formula for the standard error to give the OP an idea of how the level of certainty depends on i and j in a simple way. To complicate the situation would make the result more complicated and the dependence I i and j would not be as clearly seen. Sep 5, 2012 at 17:22
• (1) It is crucial to note this is not a poll: it's the actual election ("there was a general election..."). (2) The reference to "$n$" instead of $2$ indicates an interest in multi-candidate elections. (3) Another critical complication occurs to me: in an election, the "population" consists of all ballots. Before all are opened, the network can only estimate the number of ballots. Doesn't that make it difficult (if not impossible) to apply the finite population correction factor?
– whuber
Sep 5, 2012 at 18:06
• Since I am getting several downvotes,would someone explain the justification for it? Sep 6, 2012 at 0:45