0
$\begingroup$

I'm having doubts about some aspects of hypothesis testing, and I would appreciated if someone could please clarify / point me to some relevant literature or posts. The ones I could find so far didn't help.

An election took place yesterday in country X. 10 million people voted. The votes are being counted today, and the results are updated in real time. Out of the first 1000 'revealed' votes, 540 are in favour of candidate A. A journalist asks: 'Will candidate A have the absolute majority (>50%) in the end?'.


Here's how I approached this (theoretical) problem.

  1. I assumed that the voting papers were being sampled completely randomly, without any geographic or other bias. As if all 10 million papers were continuously mixed in a huge box, and picked one by one by a robotic arm.

  2. The probability distribution that seemed to fit this situation was the hypergeometric distribution (random sampling without replacement). $N_A$ = number of votes for A in the total 'population'; $N = 10000000$, $n_A$ = number of votes for A in a random sample of $n$ voting papers.

    $P(n_A \ votes \ for \ A \ in \ sample \ of \ size \ n)=\frac {\binom {n} {n_A} \cdot \binom {N-n} {N_A - n_A}} {\binom N {N_A}}$

Q1: Was this choice correct?

  1. As for the null hypothesis, it seemed to me that $H_0 : N_A = 5000000$ should be used, the alternative hypothesis being the journalist's question $H_1 : N_A > 5000000$.

Q2: Is it legitimate to use null and alternative hypotheses that are not strictly complementary (i.e. whose union hasn't probability 1)?

(I didn't see how the probability at point 4 below could be calculated if I didn't use an equation for the null hypothesis).

  1. Using R, I calculated the probability of rejecting $H_0$ when it's true, i.e. to find 540 or more votes for A in a sample of 1000 voting papers, under the null hypothesis.

1-phyper(539,5000000,5000000,1000)
[1] 0.006219881

Q3: Although I get the point that the probabilities of finding 540, 541, ..., 1000 votes for A must be summed, hence my use of the cumulative phyper rather than dhyper, what is the theoretical justification for this? Our sample is only one, and contains exactly 540 votes for A: how do we decide that the probability of rejecting the null hypothesis depends on all the probabilities from 540 to 1000? Why not from 0 to 540? Based on where the expected value (in this case 500) is placed compared to the sample?

  1. My conclusion was to reject the null hypothesis with a significance level $p < 0.01$. The observation of 54% votes for A in a random sample of 1000 votes would occur about 0.6% of the time, if candidate A actually had 50% votes in the total population of 10 million.

Q4: Is this a correct conclusion? I'm still uneasy about the formulation of $H_0$ as an equation, because it doesn't seem to answer exactly what the journalist asked. Candidate A would not have the absolute majority even if he had less than 5 million votes.

Q5: After some struggle, I finally understood the (rather subtle) difference between 'rejecting or not rejecting the null' and 'accepting or not accepting the alternative'. So I am fine with the idea that we can only tell the journalist that we reject the hypothesis that A has no absolute majority, because we decided that the probability of being wrong in doing so is smaller than 1%.
Would it be legitimate in this case to calculate the probability that we are right in rejecting the null (i.e. the probability of rejecting the null when it's false, which I believe is called 'power')? From many discussions I read about power it appears that this should only be evaluated before a test, not a posteriori.

Some aspects of this are still quite nebulous in my mind, so it may be that I've written some nonsense; that's exactly why I'm asking for your help!

$\endgroup$
  • $\begingroup$ Assumption (1) is far from the truth. Thus, this might serve as an exercise in Binomial sampling, but please don't take it as having any bearing on predicting elections! Your individual questions are all addressed extensively in other threads on this site, so please search. For instance, Q3 is answered at stats.stackexchange.com/questions/31, answers to Q2 are found at stats.stackexchange.com/search?q=composite+null+hypothesis, Q4 can be checked using a Binomial or Normal approximation, and questions about power can be found at stats.stackexchange.com/search?q=power. $\endgroup$ – whuber Mar 13 '18 at 22:37
  • $\begingroup$ Thank you @whuber . As I said, I did search for answers and could not find anything that matched, and from your comment I can see now that only an expert could see the connection in some cases. E.g. the title of your first link 'What is the meaning of p values and t values in statistical tests?' would have never in a million years told me that this was the answer to my Q3. I will now patiently study the various posts you indicated. $\endgroup$ – user6376297 Mar 14 '18 at 11:41
  • $\begingroup$ I wish there were an easier way to perform searches: we need better semantic tagging. Typically, though, once you learn a few key words like "p value" the system works pretty well. $\endgroup$ – whuber Mar 14 '18 at 13:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.