I am reading the article I’m A Stats Prof. Here’s Why Nate Silver’s Model Was All Over The Place on a news website (not an academic publication).
The author (Dale Rosenthal, Clinical Assistant Professor of Finance, University of Illinois at Chicago) is trying to articulate a critique of Nate Silver's presidential election modeling. His first point has to do with model formulation:
538 should be modeling each state’s race with a generalized linear model: either a multinomial model to estimate the probabilities of Clinton, Trump, Johnson, McMullin, and Stein each winning that state or a logistic-link binomial model for Trump vs Clinton. Those models were created for these sorts of scenarios. It’s a little bit of work to use these: you have to input the number of respondents in favor of each candidate instead of just sticking in the reported percentages. However, that would have the added advantage of not trusting any given poll’s claims of uncertainty.
While Nate Silver doesn’t spell it out on his site, he appears to be using either a linear regression or a logistic regression. Since the logistic regression is a better choice, I’ll assume he is using that. Some people might confuse logistic regression and a binomial GLM with a logistic [OP note: I think he means logit] link, but they aren’t the same. The difference is in how they handle the uncertainty of unusual events (i.e. likely landslides). This is because a binomial [OP note: I think he means bernoulli] random variable with probability of success p has a variance of p*(1-p). In other words: a race that is nearly tied is much more sensitive to all the inputs than a race that is likely to be a landslide. For example, Reagan would have had to screw up hugely to have lost to Mondale ― while even a small screw-up for W might have handed the win to Gore.
A binomial GLM with a logistic link is built to that sort of variation in sensitivity. Logistic regression is not built to handle that. Because logistic regression doesn’t handle that variation in sensitivity, it tends to be biased for events which are estimated to be rare. Since most polls and meta-pollsters are estimating a Trump win an very unlikely, this suggests that Silver’s model form is likely biasing his results.
I always thought I was doing "logistic regression" when I invoke the GLM:
glm(formula, family=binomial(link = "logit")). But the author seems to have something different in mind.
Somewhat related questions:
- US Election results 2016: What went wrong with prediction models?
- What is the difference between logistic and logit regression?
It sounds like what the author is trying to say is that vote counts should be modeled as binomial random variables rather than state outcomes as bernoulli random variables. Is that interpretation correct, or what exactly is the author trying to say?