5
$\begingroup$

The interpretation of the p-value is a very difficult thing because what is legit and what is not is very close together. The p-value is the probability of the resulting test statistic if the null hypothesis $H_0$ is true; that means $\Pr(\mathrm{data}|H_0)$. A lot of people make the mistake of confusing this with $\Pr(H_0|\mathrm{data})$, which is wrong. Now to my question: I stumbled upon an online independent t-test and the result was a p-value of 0.20. The given interpretation was the following: "There is a 79.76% chance the proportions are different."

My question: is this interpretation legit? Or is it the same confusion as mentioned above (e.g. it describes the probability of $H_0$). Thanks in advance!

$\endgroup$
3
  • 4
    $\begingroup$ "the p-value is the probability of the resulting test statistic if h0 is true". That's not correct. "[...] the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true". (from Wikipedia) $\endgroup$ – boscovich May 31 '13 at 13:45
  • 3
    $\begingroup$ This interpretation is wrong. As you said: the $p$-value is the probability of observing a $t$-value at least as extreme as the one you found if the null hypothesis were true. $\endgroup$ – COOLSerdash May 31 '13 at 13:46
  • $\begingroup$ Because the test was already performed, the conclusion of whether the means are different or not different should not have a probability attached to it. You either reject, or not reject, cannot 20.24% reject. $\endgroup$ – Penguin_Knight May 31 '13 at 13:47
9
$\begingroup$

The null hypothesis in this case is that the proportions are equal. One alternative hypothesis that the two are not equal. The statement "there is a 79.76% chance that that the proportions are different" is a statement about the probability of the alternative hypothesis. This is not the correct interpretation of the p-value. Recall that a p-value is the probability that, purely due to the features of random sampling under the null model, you would observe a test statistic at least as extreme as that calculated for your particular sample. All the p-value tells you is the probability of observing the data, given the null model.

$\endgroup$
1
  • 3
    $\begingroup$ Close, but you're forgetting about the test statistic, which partitions the sample space & orders those partitions: "...under the null model you would observe a test statistic at least as extreme as that calculated for your particular sample" would do it. $\endgroup$ – Scortchi - Reinstate Monica May 31 '13 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.