The setting is as follows. We have a population of size $N$. Each subject has two properties $A$ and $B$, which can be either true or false.
The question is: if for a random subject $A$ holds, is the chance that $B$ holds greater, then that for another random subject $B$ holds?
So I want to compare: $H_0 : P(B|A) \le P (B) $ and $H_1 : P(B|A) > P(B)$.
Moreover, we can only question $100$ subjects. We know that the classes $A$ and $B$ are skewed, i.e. $P(A) < 0.001$ and $P(B) < 0.001$, but we don't know the exact values of $P(A)$ and $P(B)$. Alse $N >> 100$.
My attempt
I first modify both hypothesis, because I expect that $P(B|A)$ won't be smaller than $P(B)$.
$H_0 : P(B|A) = P (B) $ and $H_1 : P(B|A) \not = P(B)$
Since we have a small number of subjects $(100)$, I suspect that we have to use Fisher's exact test instead of Pearson's chi-squared test.
Now, I found that one of the assumptions is that the sampling we do is random. See Fisher's exact test principles. This poses a problem, because the classes are soo skewed, with random sampling (if I understand it correctly?) it is very likely that we end up with a contingency table that looks like:
$ \begin{array}{l|l|l|} & A & !A \\\hline B & 0 & 0 \\\hline !B & 0 & 100 \\\hline \end{array} $.
An alternative would be to violate the random sampling. We would some select subjects for which we already know that either $A$ or $B$ knows. But as we violate an assumption are test would not be valid.
I hope that someone can help me with independence tests in case of strongly skewed classes. And that someone can give comments on my attempt and understanding of the subject.
Kind regards,
