As the election is a one time event, it is not an experiment that can be repeated. So exactly what does the statement "Hillary has a 75% chance of winning" technically mean? I am seeking a statistically correct definition not an intuitive or conceptual one.
I am an amateur statistics fan who is trying to respond to this question that came up in a discussion. I am pretty sure there's a good objective response to it but I can't come up with it myself...
All the answers so far provided are helpful, but they aren't very statistically precise, so I'll take a shot at that. At the same time, I'm going to give a general answer rather than focusing on this election.
The first thing to keep in mind when we're trying to answer questions about real-world events like Clinton winning the election, as opposed to made-up math problems like taking balls of various colors out of an urn, is that there isn't a unique reasonable way to answer the question, and hence not a unique reasonable answer. If somebody just says "Hillary has a 75% chance of winning" and doesn't go on to describe their model of the election, the data they used to make their estimates, the results of their model validation, their background assumptions, whether they're referring to the popular vote or the electoral vote, etc., then they haven't really told you what they mean, much less provided enough information for you to evaluate whether their prediction is any good. Besides, it isn't beneath some people to do no data analysis at all and simply draw a precise-sounding number out of thin air.
So, what are some procedures a statistician might use to estimate Clinton's chances? Indeed, how might they frame the problem? At a high level, there are various notions of probability itself, two of the most important of which are frequentist and Bayesian.
In a frequentist view, a probability represents the limiting frequency of an event over many independent trials of the same experiment, as in the law of large numbers (strong or weak). Even though any particular election is a unique event, its outcome can be seen as a draw from an infinite population of events both historical and hypothetical, which could comprise all American presidential elections, or all elections worldwide in 2016, or something else. A 75% chance of a Clinton victory means that if $X_1, X_2, …$ is a sequence of outcomes (0 or 1) of independent elections that are entirely equivalent to this election so far as our model is concerned, then the sample mean of $X_1, X_2, …, X_n$ converges in probability to .75 as $n$ goes to infinity.
In a Bayesian view, a probability represents a degree of believability or credibility (which may or may not be actual belief, depending on whether you're a subjectivist Bayesian). A 75% chance of a Clinton victory means that it is 75% credible she will win. Credibilities, in turn, can be chosen freely (based on a model's or analyst's preexisting beliefs) within the constraints of basic laws of probability (like Bayes's theorem, and the fact that the probability of a joint event cannot exceed the marginal probability of either of the component events). One way to summarize these laws is that if you take bets on the outcome of an event, offering odds to gamblers according to your credibilities, then no gambler can construct a Dutch book against you, that is, a set of bets that guarantees you will lose money no matter how the event actually works out.
Whether you take a frequentist or Bayesian view on probability, there are still a lot of decisions to be made about how to analyze the data and estimate the probability. Possibly the most popular method is based on parametric regression models, such as linear regression. In this setting, the analyst chooses a parametric family of distributions (that is, probability measures) that is indexed by a vector of numbers called parameters. Each outcome is an independent random variable drawn from this distribution, transformed according to the covariates, which are known values (such as the unemployment rate) that the analyst wants to use to predict the outcome. The analyst chooses estimates of the parameter values using the data and a criterion of model fit such as least squares or maximum likelihood. Using these estimates, the model can produce a prediction of the outcome (possibly just a single value, possibly an interval or other set of values) for any given value of the covariates. In particular, it can predict the outcome of an election. Besides parametric models, there are nonparametric models (that is, models defined by a family of distributions that is indexed with an infinitely long parameter vector), and also methods of deciding on predicted values that use no model by which the data was generated at all, such as nearest-neighbor classifiers and random forests.
Coming up with predictions is one thing, but how do you know whether they're any good? After all, sufficiently inaccurate predictions are worse than useless. Testing predictions is part of the larger practice of model validation, that is, quantifying how good a given model is for a given purpose. Two popular methods for validating predictions are cross-validation and splitting the data into training and testing subsets before fitting any models. To the degree that the elections included in the data are representative of the 2016 US presidential election, the estimates of predictive accuracy we get from validating predictions will inform us how accurate our prediction will be of the 2016 US presidential election.
When statisticians want to predict a binary outcome (Hillary wins vs Hillary does not win), they imagine that the universe is tossing an imaginary coin - Heads, Hillary wins; tails, she loses. To some statisticians, the coin represents their degree of belief in the outcome; to others, the coin represents what might happen if we reran the election under the same circumstances over and over. Philosophically speaking, it's hard to know what we mean when we speak of uncertain future events, even before we drag numbers into it. But we can look at where the number comes from.
At this point in the election, we have a sequence of poll results. These are of the form: 1000 people were polled in, say, Ohio. 40% support Trump, 39% support Hillary, 21% are undecided. There would be similar polls from previous elections for the respective Democratic, Republican (and other trace party) candidates. For previous years, there are also outcomes. You might know that, say, candidates with 40% of the vote in a poll in July, won 8 out of the 10 previous elections. Or the results might say, in 7 out of 10 elections, Democrats took Ohio. You might know how Ohio compares to Texas (perhaps they never choose the same candidate) - you might have information on how the undecided vote breaks down - and you might have interesting models of what happens when a candidate begins to "surge". They might also look at who tends to actually get out and vote, and what happens if it's snowing on The Day.
So when you take previous elections into account, you can say that the election coin has already been tossed a number of times. The same election is not being rerun every 4 years, but we can pretend that it sort of is. From all this information, the pollsters build complex models to predict the outcome for this year.
Hillary's 75% chance of winning is relative to our state of knowledge "today". It's saying that a candidate with the kind of poll results she has "now", in the states that she has them, and given the trends in her polls throughout the campaign, wins the election in 3 election years out of 4. A month from now, her probability of winning will have changed, because the model will be based on the state of polls in August.
The US hasn't had a statistically large number of elections in its history, much less since polling began. Nor can we be sure that polling trends from, say, the 70's, still apply. So it's all a bit dodgy.
The bottom line is that Hillary should begin working on her inauguration speech.
When statisticians say this they are not referring to the margin of victory or the share of the vote. They are running a large number of simulations of the election and counting what percentage of the vote each candidate gains. For many robust presidential models they have forecasts for each state. Some are close and if the race is run multiple times, both candidates could win. Because prediction intervals many times overlap a margin of victory of 0, it is not a binary response but instead a simulation will tell us more precisely what to expect.
FiveThirtyEight's methodology page may help understand a little more what is under the hood: http://fivethirtyeight.com/features/a-users-guide-to-fivethirtyeights-2016-general-election-forecast/
There's an episode of freakonomics radio that is very relevant to this question (in general, not in the specifics of en election). In it, Stephen Dubner interviews the lead of a project from a United States defense agency to determine the best way to forecast global political events.
It [also] helps a lot to know more about politics than most people do. I would say they’re almost necessary conditions for doing well. But they’re not sufficient, because there are plenty of people who are very smart and close-minded. There are plenty of people who are very smart and think that it’s impossible to attach probabilities to unique events.
Then they discuss what not to do
if you ask those types of questions, most people say, “How could you possibly assign probabilities to what seem to be unique historical events?” There just doesn’t seem to be any way to do that. The best we can really do is, use vague verbiage, make vague-verbiage forecasts. We can say things like, “Well, this might happen. This could happen. This may happen.” And to say something could happen isn’t to say a lot.
Then the episode goes into the methodologies that the most successful forecasters used to estimate these probabilities, advocating an informal Bayesian approach
So, knowing nothing about the African dictator or the country even, let’s say I’ve never heard of this dictator, I’ve never heard of this country, and I just look at the base rate and I say, “hmm, looks like about 87 percent.” That would be my initial hunch estimate. Then the question is, “What do I do?” Well, then I start to learn something about the country and the dictator. And if I learn that the dictator in question is 91 years old and has advanced prostate cancer, I should adjust my probability. And if I learn that there are riots in the capital city and there are hints of military coups in the offing, I should again adjust my probability. But starting with the base-rate probability is a good way to at least ensure that you’re going to be in the plausibility ballpark initially.
The episode is called How to Be Less Terrible at Predicting the Future, and is a very fun listen. I encourage you to check it out if you're interested in this sort of thing!
The 2016 election is indeed a one time event. But so is the flip of a coin or the toss of a die.
When someone claims they know a candidate has a 75% chance of winning they are not predicting the outcome. They are claiming they know the shape of the die.
The outcome of the election can't invalidate this. But if the model they use to arrive at 75% is tested against many elections it could be shown to have limited predictive value. Or it may be born out as valuable.
Of course, once a valuable predictor is known to the candidates they can change their behavior and the model can be made irrelevant. Or it can be blown all out of proportion. Just look at what happens in Iowa.
When someone says that "Hillary has a 75% chance of winning", they mean that if you offered them a bet where one person gets 25 dollars if Hillary wins and the other person gets 75 dollars if Hillary does not win, they would consider that a fair bet and have no particular reason to prefer either side.
These percentages typically come from prediction markets. These summarize all the information available and typically outperform analytical methods of predicting most events.
Prediction markets offer people the opportunity to wager on whether or not a particular event will occur. The payoffs are set by negotiation between the people on both sides of the proposition. Generally, people who have special knowledge about a proposition will be try to leverage that knowledge to make money, which has the side effect of leaking that information.
For example, suppose there's a prediction market on whether a particular celebrity will live until the end of this year. The public knows the celebrity's age and anyone can look up the basic probability that the celebrity will die by the end of the year. If that was all that was known, you would expect people to be willing to bet on one side or the other of this proposition at roughly that probability.
Now, suppose someone knew that celebrity was in poor health but was concealing it. Or even say lots of people knew that that celebrity's family had a history of heart disease that would reduce their odds of surviving. The people with that information will be willing to take one side of that proposition, causing the rate to adjust just as buyers push a stock price up and sellers push it down.
In other words, when the odds are too low, people looking to profit push them up. And when they are too high, people looking to profit push them down. The price of the bet ultimately reflects the collective wisdom of everyone on the odds of the proposition occurring just as all prices reflect collective wisdom on the costs and values of things.
The key question is how do you assign a probability to a unique event.the answer is that you develop a model by which it is no longer unique. I think an easier example is what is the probability of the president dying in office? You may view the president as a person of a certain age, as a person of a certain age and sex,. Etc... each model gives you a different prediction ...a priori there is no correct model..it is up to the statistician to select which model is most appropriate.
Given the polls show a very tight race, the 75% may or may not be accurate.
You are asking what it means, not how did they calculate this. The implication is that (if we ignore anyone else except Clinton and her one major opponent) that you would need to bet \$3 to get a \$4 return if she wins. Alternately, a \$1 bet on the other runner would return $4 if he wins.
My answer makes a small distinction, between the actual chance for either candidate to win, and what people (gamblers, or odds makes) are expecting. I suspect that when you see numbers like this, e.g. 75%, you are seeing the odds makers numbers, when you see 49 to 48%, you are seeing poll results.
If they're doing it right, something happens approximately three-fourths of those times when they say it had a 75% chance of happening. (or more generally, the same idea adapted over all percentage forecasts)
It is possible to ascribe more meaning than that depending on our philosophical opinions and how much we believe the models, but this pragmatic point of view is something of a lowest common denominator — at the very least, statistical methods try (although possibly as a side effect rather than directly) to make forecasts obeying this pragmatic point of view.
