With the recent US presidential election, there seems to be a plethora of election "modelers" (e.g. 538). Given their "accuracy" of predicting the election outcome, I have come up with a couple of criticisms of what I perceive to be deficiencies in election modeling.

  1. To have a valid computational model (e.g. in the physical sciences), one must do extensive validation. This means comparing your model to known physical cases with known outcomes. Without validation, computational models are irrelevant because there is no measure of either their accuracy or precision. Since demographics change and elections are held only once, this seems particularly challenging to election modeling.

  2. The election is held once with a binary outcome. There is no way to measure the outcome of the election more than once. It is unclear how one can attach a "chance of winning" to a single measurement of the state of the system and interpreting what relevance that even has.

QUESTION : Given the lack of validation, how is modeling elections (like 538 does) a valid approach? Likewise given that elections are held only once, how is the percent "chance of winning" supposed to be interpreted?

  • 1
    $\begingroup$ This seems like a better question for Cross Validated. $\endgroup$
    – jwodder
    Dec 14 '16 at 13:28
  • 1
    $\begingroup$ As much as I hate to say it, this is a great political science question - but a poor fit for politics.SE. Currently this is not within our scope. $\endgroup$ Dec 14 '16 at 14:30
  • $\begingroup$ As suggested by a flag I will migrate this to cross validated. Feel free to send it back when it doesn't fulfill your quality criteria. $\endgroup$
    – Philipp
    Dec 14 '16 at 14:51
  • 2
    $\begingroup$ @Aksakal 538 was predicting a 70% chance for Hillary (some people may've misinterpreted this to mean 70% of people were voting for Hillary), not 90%. And, the fact that Trump won doesn't invalidate the model. A 30% chance isn't a 0% chance. Minority probabilities have to occur, otherwise it's not a model, it's a flat out prediction. The general answer here is that you look at sample errors in the polls and use those to compute total probability. I can provide more details if the linked paper above isn't enough. $\endgroup$
    – user1566
    Dec 14 '16 at 16:18
  • 1
    $\begingroup$ @barrycarter, 70% was a day before the election, after the 3rd debate they were giving 90% chance to Clinton. the model is not invalid, it's useless $\endgroup$
    – Aksakal
    Dec 14 '16 at 16:37

You are thinking in a deterministic physics way. I like it. But it cannot be applied to politics (yet?). Instead, think in a probabilistic way.

Maybe you heard one day that whenever a poll is done, there is a margin of error depending on the number of people interrogated, and the population. This margin of error is just a small representation of the uncertainty represented. The result of a poll in state A is not "30 % of people want to vote for X and 70% want to vote for Y, error 3 %". It is " 30 % of the folks we picked wanted to vote for X, the others for Y ". Now, what should we do about that?

Picture a big bag containing 1000 white and black balls. You want to know if there are more white or black balls. You don't want to pick every ball in the bag, so you pick 10, let us say 6 are white.

The result you get is a "probability density". That is, for each repartition of balls, the chance that it was this one and that you picked 6 whites:

  • "what is the probability that there are 100 black balls knowing that I picked 6 whites ? "

  • "what is the probability that there are 101 black balls knowing that I picked 6 whites ? "

  • ...

The results of a poll are similar : it is an estimation, for each vote repartition, of its chance of happening KNOWING your poll results. Before the poll, any vote repartition was equally possible. After the poll, it is not the case anymore. Some are more likely.

This is what is meant by "chances of winning" : it is the probability that at least half of the voters vote for X, knowing your poll results. With the american electoral college, the analysis your link presents is interesting, since it combines the results of the different states for you.

Some of the problems with this kind of model (among others) are:

  • People change their minds

  • Some people avoid polls

  • Some people lie to polls (there is this long known observations of pollers : people voting for alt-right don't confess it to the poller. This leads to a "correcting factor" )

  • context changes (social, cultural, economical, ...)

  • people don't know how to read polls (ask Winston Churchill his opinion about them). The question, method of polling, and plenty of other factors have to be taken into account.

  • ...

(undecided voters and other candidates can just be represented with another color, they are not really a problem)

Side note : If you want to learn about something close to validated social science, have you ever heard about psychohistory ?

  • 2
    $\begingroup$ In statistics, additionally to the error margin it is the (often overlooked) confidence level; usuallly 95% is used so it is not explicitly stated. A 3% error actually means "we are 95% sure that the error will not be greater than 3%", which gives a 5% possibility of a bigger error. And that is when you apply to "stable" things like the color of balls and not to something that may change as the opinion of people. $\endgroup$
    – SJuan76
    Dec 14 '16 at 14:32

Not the answer you're looking for? Browse other questions tagged or ask your own question.