1
$\begingroup$

After the recent American election, a list of 14,000 "dead voters" went viral online. When CNN picked 50 of these voters at random, they found that:

Of the 50, 37 were indeed dead and had not voted, according to the voter information database. Five people out of the 50 had voted -- and they are all still alive, according to public records accessed by CNN. The remaining eight are also alive but didn't vote.

There are two possible conclusions here (one intuitively more likely than the other). The first is that all 50 of the CNN random picks were chance mistakes (the null hypothesis), the second is that the list was wrong.

How would I go about calculating the likelihood that the list was wrong? How do I attach a p-value to this outcome?

$\endgroup$
1
  • 1
    $\begingroup$ You need to be clearer about what you mean by "the list was wrong." What do you intend this to mean? Indeed, what constitutes a "mistake"?? $\endgroup$
    – whuber
    Jan 12, 2021 at 14:43

2 Answers 2

3
$\begingroup$

Like Three Diag in his very good answer let's consider this a binomial problem with p being the probability of each single member of the list being dead and having cast a vote. The Bayesian question would be how to estimate our best guess of that probability and for binomial problems that is expecially easy because we can use a conjugated, in this case a beta distribution.

I have no idea how this list came together. It may be taken out of thin air or be a very precise report. So we have a rare situation where a flat prior seems in order. For convenience / conjugation let's call it a $Beta(1, 1)$ prior:

enter image description here

Now with the magic of conjugation we can update that with n = 50 to a $Beta(1,51)$ assumption:

enter image description here

This is usefull because we can answer all kinds of questions with that. You asked

How would I go about calculating the likelihood that the list was wrong?

Now, what is a wrong list? I tend to say: If 20% of more of that list are true voter fraud, I'd consider that a scandal worth noting here one the other side of the atlantic ocean. But you could set any other limit.

The probability of the probability being smaller then 20% is the area from 0 to 20% which is conveniently implemented in R as pbeta(x, 1, 51).

> pbeta(.2, 1, 51)
[1] 0.9999886

Given a flat prior, the chance of less then 20% true-fraud values is 99.999%.

Now of course that is somewhat dependend on the prior. If some Trumpists started out with a 10 to 1 prior of $Beta(1,10)$ as in enter image description here

They would come up with a largely different posterior probability distribution: enter image description here But even if you started with a 10-to-1 Beta-Prior the probability of p < 20% is 77.7%.

$\endgroup$
2
$\begingroup$

Let us say you have a large list of supposedly dead voter, you pick one at random and check whether they are actually dead and actually voted, in which case you mark a one, otherwise there must be a mistake in the list and you mark it with a zero.

This is a bernoulli variable:

$$ X_i = \begin{cases} 0, \text{with prob. }\ p \\ 1, \text{ with prob.}\ 1-p \end{cases} $$

where $p$ is the true rate of mistakes in the list.

We say the list is ``right'' if $1-p \approx 1$, that is if the fraction of mistake $p\approx0 $ is small.

What is the chance of observing 50 mistakes out of 50? And what kind of error rate should we need to make this a reasonably rare outcome (lending evidence to the possibility that the list is indeed right?), say something that might might happen 5% of the time?

Well, the counting of "confirmed dead voters" is a binomial

$$ S = \sum_{i=1}^{50} X_i \sim Bin(50,1-p) $$

the chance of observing 50 mistakes is given by

$$ P(S = 0) = \binom{50}{0} p^{50}(1-p)^0 = p^{50}. $$

This is already quite explicative, if we want to sometimes, even just 5% of the time to see the event happening, we need

$$ p^{50} = 0.05 \rightarrow p = 0.05^{1/50}\approx 0.94 $$

an error rate of 94% is needed!

Formalise this as a p-value by setting a null hypothesis, the error rate is low if $p$ is low, say 1%:

$$ H_0 : p =0.01 $$

the p-value is simply the probability of a statistic more extreme than the one we got which is 50, which happens to be the most extreme possible, it's just then

$$ P(S = 0 \vert H_0\ true) = 0.01^{50} = 1^{-100} $$

and with this very small p-value we would say we have some evidence to reject the null that "the list is right".

Notice how not even a list which contains 80% of mistakes is likely to turn up such a statistic, the p-value for that null-hypothesis

$$ P(S = 0 \vert H_0\ true) = 0.9^{50} = 0.0051 $$

is still less then 1% which is considered extremely strong evidence against the null.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.