Scenario

Suppose that there is the population of Raspberry Pies (RPs).

Suppose that you sample 5 of those RPs, and after 1 month, you find that 3 of them have some problem (faulty chip). So the sample probability that RPs have a problem is 3/5.

Then suppose that you find a magical spray can that says "if you spray this on RPs, they shall be purified from all chip illnesses".

Suppose that you sample another 5 RPs, except that you immediately spray them with that magical purification spray.

Suppose that you wait 1 month, and: all of the 5 recently sampled RPs have absolutely no faulty chips in a month (which is an improvement over the previously sampled RPs that got the faulty chip in 1 month).

Hypothesises

• H0 - the magical spray is a scam. It makes no difference, and the observed enhancement is due to sheer dumb luck.

Question

• Suppose that H0 is true, what is the probability to observe the above scenario?

• Suppose that I tell you that all quantities of RPs that I have samples, were 1000 more instead of just 5. What difference would this information make to your answer with respect to the first question?

• Is my attempt below exactly what's called Fisher's exact test?

• Is my observation OBSERVATION 1 exactly likelihood maximization?

My attempt

If we suppose that H0 is the case, then it means that having all the 5 RPs not die in a month after getting magic-sprayed is due to sheer dumb luck.

That means that, the probability that a RP fails after a month is independent of whether it got magic-sprayed. If I abuse notation, that means: $\Pr(\text{RP dies in 1 month}) = \Pr(\text{RP dies after 1 month}|\text{got_magic_sprayed})$.

REMARK 1: Therefore it means that, if H0 is true, then the probability of observing a dead RP is just $\Pr(\text{RP dies after 1 month}) = 3/5$.

GUESS 1: I guess H0 must also imply that the death of any RPs is independent of the state other RPs. (correct me please).

Using REMARK 1: and GUESS 1, it seems that (if H0 is true) sampling any 5 RPs with 3 dead RPs after 1 month is:

• Probability of this: dead, dead, dead, ok, ok.
• Plus that of this: dead, dead, ok ,dead, ok.
• Plus ... all combinations of 3 deads and 2 oks.

We have ${5 \choose 3}$ many such combinations, with each combination having the probability to occur (under H0) $(3/5)^3 (1-3/5)^2$. I.e. ${5 \choose 3} (3/5)^3 (1-3/5)^2 = 0.3456$.

Now, applying the same logic, what is the probability of sampling 5 RPs such that ALL of them die after 1 month assuming H0 holds? Here is this: ${5 \choose 5} (3/5)^5 (1-3/5)^0 = 0.216$.

ANSWER 1: Now, same but zero deads, all OK: ${5 \choose 0} (3/5)^0 (1-3/5)^5 = 0.01024$. Oh my godness, did you see that? Since $p \approx 0.01$ is too tiny (less than than $0.05$, I reject H0, which necessarily implies that the magic spray can is true!

OBSERVATION 1: assuming that H0 is true, the measured probability is maximum only if number of dead RPs is 3, and ok RPs is 2.