1
$\begingroup$

I have a question related to hypothesis testing. I cannot seem to find error in my reasoning.

Initial question: We have 230 patients: 157 males and 73 females. 142 males are sick and 69 females are sick. Can we say that females are sick "statistically significantly" more?

My method:
Assume that females and males come from binomial distributions. Females are sick with probablity $p_\text{female}$, males - $p_\text{male}$.

We want to test that $p_\text{female} = p_\text{male} = p_\text{overall}$

Given that null hypothesis is true, the probability of observing such "extreme" results (so many females and so little males being sick) is:

cdfbinom(69, 73, prob=211/230)) * (1-cdfbinom(142, 157, prob=211/230)

Which is equal to 0.04351274, therefore we reject the hypothesis that they are the same.

I cannot understand where is my mistake, as all of the other methods: chi-square test, t-test, even running logistic regression where I include gender indicator as a variable, do not reject the null hypothesis.

$\endgroup$
1
$\begingroup$

You are only considering one-sided extremities.

In your probability statements, you should also include the 'extreme cases' of having 'few women' and 'many men'. This will surely raise the probability total, and will likely be above your default threshold (0.05) since it's already close.

$\endgroup$
  • $\begingroup$ i would use a chisq test to test independence of the factors sex and sick.. Is this a right approach? $\endgroup$ – user974514 Mar 25 '13 at 17:06
  • $\begingroup$ Thanks for your answer, Nick. However, the problem is that the question is asking if females are sick "more often", not "if the sickness rates differ" - is my method of testing correct then? Also, another confusing part is that in all the other tests I get p-values around 0.3, and in this case even if I would include "both extremities" the discrepancy between other tests is still rather big... $\endgroup$ – Vainius Mar 26 '13 at 13:01
  • 1
    $\begingroup$ OK, I looked into this a little deeper (but only a little). I was misled on the one-sided part because you compare your result to a chi-square test, which is, to my knowledge, always two-sided (and thus will always have a bigger p-value). I've no clue which t-test you tried to perform, so I'll stay away from that. Default outputs from most logistic regressions will also show two-sided p-values. $\endgroup$ – Nick Sabbe Mar 26 '13 at 14:47
  • 1
    $\begingroup$ As for your own suggested p-value: at the very least, it's not so obvious that you can just plug in your estimate (211/230) for the population proportion in the cumulative probabilities. $\endgroup$ – Nick Sabbe Mar 26 '13 at 14:50
  • $\begingroup$ I have also tried pluging 'optimal' value (in sense that it gives maximal p_value) for it and it was very close to 211/230. Yes, but it is still not obvious if I can call the number I get a "p-value"... $\endgroup$ – Vainius Mar 26 '13 at 15:43
1
$\begingroup$

There are several problems with this approach, here are two:

  • Each part of your computation is basically the p-value from a one-tailed one-sample test that the population proportion is different from 211/230. This does not directly test for the difference between groups and also ignores the uncertainty in this number (i.e. 211/230 is not some theoretical value or a priori hypothesis, it's something you computed on the same data).
  • You cannot combine $p$-values in this way. One intuition to show this: The cumulative distribution function and $p$-values are bounded between 0 and 1; if you multiply several $p$-values together you will inevitably end up with increasingly smaller numbers and eventually cross whatever threshold you set. Consequently, you could take any sample (including pure noise or random data in which the null hypothesis is true by construction), slice it in small batches, run a test on each batch and multiply the $p$-values together to obtain a very small number. What could it possibly mean?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.