Hypothesis testing using different methods

Question

I have a question related to hypothesis testing. I cannot seem to find error in my reasoning.

Initial question: We have 230 patients: 157 males and 73 females. 142 males are sick and 69 females are sick. Can we say that females are sick "statistically significantly" more?

My method:
Assume that females and males come from binomial distributions. Females are sick with probablity $p_\text{female}$, males - $p_\text{male}$.

We want to test that $p_\text{female} = p_\text{male} = p_\text{overall}$

Given that null hypothesis is true, the probability of observing such "extreme" results (so many females and so little males being sick) is:

cdfbinom(69, 73, prob=211/230)) * (1-cdfbinom(142, 157, prob=211/230)

Which is equal to 0.04351274, therefore we reject the hypothesis that they are the same.

I cannot understand where is my mistake, as all of the other methods: chi-square test, t-test, even running logistic regression where I include gender indicator as a variable, do not reject the null hypothesis.

Answer 1

You are only considering one-sided extremities.

In your probability statements, you should also include the 'extreme cases' of having 'few women' and 'many men'. This will surely raise the probability total, and will likely be above your default threshold (0.05) since it's already close.

Answer 2

There are several problems with this approach, here are two:

• Each part of your computation is basically the p-value from a one-tailed one-sample test that the population proportion is different from 211/230. This does not directly test for the difference between groups and also ignores the uncertainty in this number (i.e. 211/230 is not some theoretical value or a priori hypothesis, it's something you computed on the same data).
• You cannot combine $p$-values in this way. One intuition to show this: The cumulative distribution function and $p$-values are bounded between 0 and 1; if you multiply several $p$-values together you will inevitably end up with increasingly smaller numbers and eventually cross whatever threshold you set. Consequently, you could take any sample (including pure noise or random data in which the null hypothesis is true by construction), slice it in small batches, run a test on each batch and multiply the $p$-values together to obtain a very small number. What could it possibly mean?

Answer 3

The way you describe the problem, you can consider a contingency table:

Health	Male	Female
Sick	142	69
Healthy	15	4

Let us call $\theta$ the underlying probability of being sick (mainly to avoid so many $p$s). Several exact tests use the binomial distribution to check he null hypotheses $\theta_{female} = \theta_{male}$ or $\theta_{male} > \theta_{female}$. You can read about Fisher's exact test or Barnard's test. You can also consider using some variation of the $\chi^2$ test if the number of observations is large.

In addition, I have recently described an exact test to check any of these hypotheses in R (the package is here). Let us set $\alpha$ to 0.05 and test both hypotheses:

---

$\theta_{female} = \theta_{male}$

library(mtest)
m.test(list(c(69, 4), c(142, 15)))
[1] 0.7113878

The null hypothesis is not rejected.

---

$\theta_{male} > \theta_{female}$

os.m.test(list(c(142, 15), c(69, 4)))
[1] 0.0101545

And the null hypothesis is rejected. In this case, this does not necessarily mean that $\theta_{female} > \theta_{male}$. As seen in the previous operation, the results are compatible with $\theta_{female} = \theta_{male}$

---