3
$\begingroup$

When explaining the reason for rejecting a null hypothesis, I sometimes see "p-value is small", and sometimes it's "for a test of size $\alpha$, we reject it when (some condition)". I was wondering if "p-value is small" is a special case of a commonly used/standard (idk how to phrase this) size $\alpha$ test?
If this is not the case, could you explain when I am supposed to use which?
Thank you!

$\endgroup$
0

2 Answers 2

1
$\begingroup$

We set some value, called $\alpha$, as our maximum tolerance for type I error rate. That is, we accept that our work could reject true null hypotheses $100\alpha\%$ of the time the null hypothesis is true. In the common situation of $\alpha=0.05$, we accept that to be $5\%$. In fact, $\alpha=0.05$ is so common that it typically is implied when no $\alpha$ is specified, and we consider p-values of $0.05$ or smaller to be “small” p-values.

Then we run the test and calculate a p-value. If $p\le\alpha$, we reject the null hypothesis in favor of the alternative hypothesis.

$\endgroup$
1
$\begingroup$

The outcome of a hypothesis test is reported in two ways:

  • The p-value is p where p is a given small number.
  • The null hypothesis is rejected at the α significance level; usually α = 0.05.

If the p-value p is smaller than α, then the null hypothesis is rejected at the α level. And if the null hypothesis is rejected, we know the corresponding p-value is < α. However, we don't know the exact p-value. It might be 0.049, it might be 0.000001.

The first statement is preferred because it presents more information (the strength of evidence against the null hypothesis). Note that we don't say that p-value is significant or not; it's enough to report the p-value since it's obvious that the null hypothesis will be rejected at all significance levels α > p.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.