9
$\begingroup$

I am reading the article I’m A Stats Prof. Here’s Why Nate Silver’s Model Was All Over The Place on a news website (not an academic publication).

The author (Dale Rosenthal, Clinical Assistant Professor of Finance, University of Illinois at Chicago) is trying to articulate a critique of Nate Silver's presidential election modeling. His first point has to do with model formulation:

538 should be modeling each state’s race with a generalized linear model: either a multinomial model to estimate the probabilities of Clinton, Trump, Johnson, McMullin, and Stein each winning that state or a logistic-link binomial model for Trump vs Clinton. Those models were created for these sorts of scenarios. It’s a little bit of work to use these: you have to input the number of respondents in favor of each candidate instead of just sticking in the reported percentages. However, that would have the added advantage of not trusting any given poll’s claims of uncertainty.

While Nate Silver doesn’t spell it out on his site, he appears to be using either a linear regression or a logistic regression. Since the logistic regression is a better choice, I’ll assume he is using that. Some people might confuse logistic regression and a binomial GLM with a logistic [OP note: I think he means logit] link, but they aren’t the same. The difference is in how they handle the uncertainty of unusual events (i.e. likely landslides). This is because a binomial [OP note: I think he means bernoulli] random variable with probability of success p has a variance of p*(1-p). In other words: a race that is nearly tied is much more sensitive to all the inputs than a race that is likely to be a landslide. For example, Reagan would have had to screw up hugely to have lost to Mondale ― while even a small screw-up for W might have handed the win to Gore.

A binomial GLM with a logistic link is built to that sort of variation in sensitivity. Logistic regression is not built to handle that. Because logistic regression doesn’t handle that variation in sensitivity, it tends to be biased for events which are estimated to be rare. Since most polls and meta-pollsters are estimating a Trump win an very unlikely, this suggests that Silver’s model form is likely biasing his results.

I always thought I was doing "logistic regression" when I invoke the GLM: glm(formula, family=binomial(link = "logit")). But the author seems to have something different in mind.

Somewhat related questions:

It sounds like what the author is trying to say is that vote counts should be modeled as binomial random variables rather than state outcomes as bernoulli random variables. Is that interpretation correct, or what exactly is the author trying to say?

Improve this question
$\endgroup$
3
  • 5
    $\begingroup$ That author must have a different definition of "logistic regresion" than I have. Write him and ask? $\endgroup$
    – kjetil b halvorsen ♦
    Nov 10, 2016 at 15:36
  • 5
    $\begingroup$ I agree with @kjetilbhalvorsen. Moreover, the whole argument does not seem very convincing for me. The author seems to be saying "I do not know what methodology they used, but if they used the one that I teach at Stats 101, then they would get it right" -- it's so easy to make such judgments post factum... Why didn't he produced his forecasts before the election and published it? If you read few things that Nate Silver wrote, you'll notice that he knows what he is talking about and he knows what logistic regression is... $\endgroup$
    – Tim ♦
    Nov 10, 2016 at 15:41
  • $\begingroup$ @Tim: Yes. Another thing, thia author find one particular problem with Nate Silverss approach, that he assingns an impossible high probability for a Trump win, so must be bad ... $\endgroup$
    – kjetil b halvorsen ♦
    Nov 10, 2016 at 15:44

2 Answers 2

Reset to default
9
$\begingroup$

This sounds like pseudostatistical gibberish to me. It may be that what he has in mind is the beta-binomial distribution, which is a way to account for greater variability in the response than 'ought' to occur with a binomial, but it's hard to say. The beta-binomial distribution would not be familiar to someone who has only taken a couple of applied statistics classes, but should not be exotic to a statistics professor.

The rest of his argument sounds like a Dunning-Kruger effect to me. That is where someone knows just a little bit about a topic, but is unaware of the breadth and depth of the issues or the potential caveats and complications, and therefore thinks that the topic is easy and obvious. The idea that the best way to forecast the election is to build one simple logistic regression model with the state polls is strikingly ignorant.

Improve this answer
$\endgroup$
0
4
$\begingroup$

Logistic regression is often taught to undergrads as a transformed response: Take a number between 0 and 1, make log-odds out of that, and then fit OLS to it. That is also what is done for logistic regression in some social sciences. Given that Nate did his undergrad in economics, it would not be unusual if he had been taught this non-GLM approach.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I never had this term used like this and I doubt Nate Silver uses such "naive" logistic regression approach. Could you provide any reference naming such approach as logistic regression? $\endgroup$
    – Tim ♦
    Nov 12, 2016 at 9:52
  • 3
    $\begingroup$ Certainly you can take a number that is bounded by 0&1 & transform it (eg, w/ the log odds, or other transformation) & then fit an OLS model, but that isn't logistic regression. I hope people aren't being taught somewhere that it is. $\endgroup$
    – gung - Reinstate Monica
    Nov 12, 2016 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.