Scenario -------- Suppose that there is the population of Raspberry Pies (RPs). Suppose that you sample 5 of those RPs, and after 1 month, you find that 3 of them have some problem (faulty chip). So the sample probability that RPs have a problem is 3/5. Then suppose that you find a magical spray can that says _"if you spray this on RPs, they shall be purified from all chip illnesses"_. Suppose that you sample another 5 RPs, except that you immediately spray them with that magical purification spray. Suppose that you wait 1 month, and: all of the 5 recently sampled RPs have absolutely no faulty chips in a month (which is an improvement over the previously sampled RPs that got the faulty chip in 1 month). Hypothesises ----- - **H0** - the magical spray is a scam. It makes no difference, and the observed enhancement is due to sheer dumb luck. Question -------- - Suppose that **H0** is true, what is the probability to observe the above scenario? - Suppose that I tell you that all quantities of RPs that I have samples, were 1000 more instead of just 5. What difference would this information make to your answer with respect to the first question? - Is my attempt below _exactly_ what's called Fisher's _exact_ test? - Is my observation **OBSERVATION 1** exactly likelihood maximization? My attempt ---------- If we suppose that **H0** is the case, then it means that having all the 5 RPs not die in a month after getting magic-sprayed is due to sheer dumb luck. That means that, the probability that a RP fails after a month is independent of whether it got magic-sprayed. If I abuse notation, that means: $\Pr(\text{RP dies in 1 month}) = \Pr(\text{RP dies after 1 month}|\text{got_magic_sprayed})$. **REMARK 1:** Therefore it means that, if **H0** is true, then the probability of observing a dead RP is just $\Pr(\text{RP dies after 1 month}) = 3/5$. **GUESS 1:** I guess **H0** must also imply that the death of any RPs is independent of the state other RPs. (correct me please). Using **REMARK 1** and **GUESS 1**, it seems that (if **H0** is true) sampling any 5 RPs with 3 dead RPs after 1 month is: - Probability of this: dead, dead, dead, ok, ok. - Plus that of this: dead, dead, ok ,dead, ok. - Plus ... all combinations of 3 deads and 2 oks. We have ${5 \choose 3}$ many such combinations, with each combination having the probability to occur (under **H0**) $(3/5)^3 (1-3/5)^2$. I.e. ${5 \choose 3} (3/5)^3 (1-3/5)^2 = 0.3456$. Now, applying the same logic, what is the probability of sampling 5 RPs such that ALL of them die after 1 month assuming **H0** holds? Here is this: ${5 \choose 5} (3/5)^5 (1-3/5)^0 = 0.216$. **ANSWER 1:** Now, same but zero deads, all OK: ${5 \choose 0} (3/5)^0 (1-3/5)^5 = 0.01024$. Oh my godness, did you see that? Since $p \approx 0.01$ is too tiny (less than than $0.05$, I reject **H0**, which necessarily implies that the magic spray can is true! **OBSERVATION 1:** assuming that **H0** is true, the measured probability is maximum only if number of dead RPs is 3, and ok RPs is 2.