5
$\begingroup$

Let's say you let the Null Hypothesis $H_0$ be that the mean volume of water in a bottle is some $\mu$, and the alternate hypothesis $H_1$ be that the mean volume is not $\mu$.

Let's assume you pick a 95% confidence interval, so $\alpha = 0.05$. That would give you a critical value (assuming a normal distribution) of 1.96.

Let's say the P-Value is less than 0.05, which would mean you reject $H_0$. But what if the test statistic is less than 1.96? In that case you would reject $H_1$, but you already rejected $H_0$ from the P-value. Does one take precedence over the other, or would this never happen?

Improve this question
$\endgroup$
2

2 Answers 2

Reset to default
15
$\begingroup$

They’re synonyms. The critical value is the value the test statistic has to reach in order for your hypothesis test to reject at your chosen level (maybe $0.05$, maybe not). If you are not getting these to agree, there is a bug in your code.

You might be stumbling on the fact that confidence intervals and hypothesis tests (usually) are inverses of one another. The p-value, it turns out, is the $\alpha$-level for which a $(1-\alpha)\%$ confidence interval has the null value as one of the endpoint.

Improve this answer
$\endgroup$
2
8
$\begingroup$

Hypothesis tests are always framed in terms of the null. In the case that the test statistic is less than the critical value, then the null fails to be rejected. When test statistic exceeds the critical value, we reject the null hypothesis.

To your point, the p value could be less than 0.05 and we could still have the test statistic be less than the critical value. This would mean our chosen $\alpha$ was smaller than 0.05, and would mean we would fail to reject the null.

Improve this answer
$\endgroup$
9
  • $\begingroup$ But what if we chose $\alpha$ to be 0.05, and the p value was less than $\alpha$ and the test statistic was less than the critical value? You're saying that this would not be possible, and the only explanation is that the $\alpha$ value chosen was actually smaller? I guess what I'm really asking is what if the two "methods" (p value vs $\alpha$ and test statistic vs critical value) of rejecting $H_0$ contradict each other? $\endgroup$
    – explodingfilms101
    Nov 26, 2020 at 1:57
  • 4
    $\begingroup$ They can never contradict one another. The p value is a function of the test statistic, the critical value is a function of alpha. Your "two methods" are two sides of the same coin. $\endgroup$
    – Demetri Pananos
    Nov 26, 2020 at 2:02
  • 2
    $\begingroup$ There are no methods to choose from. There is only one procedure. $\endgroup$
    – Demetri Pananos
    Nov 26, 2020 at 2:42
  • 1
    $\begingroup$ @JDL you can have a p value less than 0.05 and still fail to reject the null. In that case, the associated tests' alpha value is less than 0.05. For example, if I chose an alpha of 0.01, then a z test with such an alpha has a critical value of 2.57. If my test yielded a test statistic of 2.3 my p value would be 0.021. I would fail to reject the null because my p value is not smaller than 0.01 all the while having my p value smaller than 0.05 $\endgroup$
    – Demetri Pananos
    Nov 26, 2020 at 15:56
  • 2
    $\begingroup$ @JDL your welcome to submit an answer you think is superior then. $\endgroup$
    – Demetri Pananos
    Nov 26, 2020 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.