3
$\begingroup$

I'm having doubts about some aspects of hypothesis testing, and I would appreciated if someone could please clarify / point me to some relevant literature or posts. The ones I could find so far didn't help.

An election took place yesterday in country X. 10 million people voted. The votes are being counted today, and the results are updated in real time. Out of the first 1000 'revealed' votes, 540 are in favour of candidate A. A journalist asks: 'Will candidate A have the absolute majority (>50%) in the end?'.

Here's how I approached this (theoretical) problem.

  1. I assumed that the voting papers were being sampled completely randomly, without any geographic or other bias. As if all 10 million papers were continuously mixed in a huge box, and picked one by one by a robotic arm.

  2. The probability distribution that seemed to fit this situation was the hypergeometric distribution (random sampling without replacement). $N_A$ = number of votes for A in the total 'population'; $N = 10000000$, $n_A$ = number of votes for A in a random sample of $n$ voting papers.

    $P(n_A \ votes \ for \ A \ in \ sample \ of \ size \ n)=\frac {\binom {n} {n_A} \cdot \binom {N-n} {N_A - n_A}} {\binom N {N_A}}$

Q1: Was this choice correct?

  1. As for the null hypothesis, it seemed to me that $H_0 : N_A = 5000000$ should be used, the alternative hypothesis being the journalist's question $H_1 : N_A > 5000000$.

Q2: Is it legitimate to use null and alternative hypotheses that are not strictly complementary (i.e. whose union hasn't probability 1)?

(I didn't see how the probability at point 4 below could be calculated if I didn't use an equation for the null hypothesis).

  1. Using R, I calculated the probability of rejecting $H_0$ when it's true, i.e. to find 540 or more votes for A in a sample of 1000 voting papers, under the null hypothesis.

1-phyper(539,5000000,5000000,1000)
[1] 0.006219881

Q3: Although I get the point that the probabilities of finding 540, 541, ..., 1000 votes for A must be summed, hence my use of the cumulative phyper rather than dhyper, what is the theoretical justification for this? Our sample is only one, and contains exactly 540 votes for A: how do we decide that the probability of rejecting the null hypothesis depends on all the probabilities from 540 to 1000? Why not from 0 to 540? Based on where the expected value (in this case 500) is placed compared to the sample?

  1. My conclusion was to reject the null hypothesis with a significance level $p < 0.01$. The observation of 54% votes for A in a random sample of 1000 votes would occur about 0.6% of the time, if candidate A actually had 50% votes in the total population of 10 million.

Q4: Is this a correct conclusion? I'm still uneasy about the formulation of $H_0$ as an equation, because it doesn't seem to answer exactly what the journalist asked. Candidate A would not have the absolute majority even if he had less than 5 million votes.

Q5: After some struggle, I finally understood the (rather subtle) difference between 'rejecting or not rejecting the null' and 'accepting or not accepting the alternative'. So I am fine with the idea that we can only tell the journalist that we reject the hypothesis that A has no absolute majority, because we decided that the probability of being wrong in doing so is smaller than 1%.
Would it be legitimate in this case to calculate the probability that we are right in rejecting the null (i.e. the probability of rejecting the null when it's false, which I believe is called 'power')? From many discussions I read about power it appears that this should only be evaluated before a test, not a posteriori.

Some aspects of this are still quite nebulous in my mind, so it may be that I've written some nonsense; that's exactly why I'm asking for your help!

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Assumption (1) is far from the truth. Thus, this might serve as an exercise in Binomial sampling, but please don't take it as having any bearing on predicting elections! Your individual questions are all addressed extensively in other threads on this site, so please search. For instance, Q3 is answered at stats.stackexchange.com/questions/31, answers to Q2 are found at stats.stackexchange.com/search?q=composite+null+hypothesis, Q4 can be checked using a Binomial or Normal approximation, and questions about power can be found at stats.stackexchange.com/search?q=power. $\endgroup$
    – whuber ♦
    Commented Mar 13, 2018 at 22:37
  • $\begingroup$ Thank you @whuber . As I said, I did search for answers and could not find anything that matched, and from your comment I can see now that only an expert could see the connection in some cases. E.g. the title of your first link 'What is the meaning of p values and t values in statistical tests?' would have never in a million years told me that this was the answer to my Q3. I will now patiently study the various posts you indicated. $\endgroup$
    – user6376297
    Commented Mar 14, 2018 at 11:41
  • 1
    $\begingroup$ I wish there were an easier way to perform searches: we need better semantic tagging. Typically, though, once you learn a few key words like "p value" the system works pretty well. $\endgroup$
    – whuber ♦
    Commented Mar 14, 2018 at 13:29

1 Answer 1

Reset to default
1
$\begingroup$

I'm working on a similar problematic and as some of the link provided by @whuber are missing here how I would approach your problem. I'm curious to see if my understanding of hypergeometric testing is sound and right, so I welcome any constructive feedback. My answers to the numbered questions above follow.

Q1 : I agree with @whuber

Q2 : It seems to me that it's the wrong H0 for your problem statement. If you use H0 = 50%, it means that you might reject it, whether the actual proportion of vote for candidate A is higher or lower than 50%. Assuming H0 <= 50% as a one sided test seems more appropriate.

Q3 : Having restated your H0 you could now use one sided confidence interval calculation as described in the following paper to get confidence interval on your estimated proportion (M): http://www.wright.edu/~weizhen.wang/paper/37-2015jasa_wang.pdf

Q4 : Will follow from the previous calculation. Your upper bound on the proportion of vote for candidate A will be your critical value. If 540 is outside the interval it means that you have an higher than chance probability that candidat A is actually ahead in the poll. And so you would reject the null hypothesis.

Q5 : Not necessarily, you can test your hypothesis and compute the power level associated. But you can also set an a priori power level that you need to be able to detect a specific effect size : http://www.real-statistics.com/sampling-distributions/statistical-power-sample/

Thank you

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks Robert; after some further study of similar problems, I am now using a Bayesian approach. Some very interesting papers e.g. this one by Colquhoun, convinced me that my previous way of looking at this was not addressing the question I wanted to answer. $\endgroup$
    – user6376297
    Commented Feb 16, 2020 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.