Assume we carry out a hypothesis test at the 5% significance level. We have an observed test statistics $t$ with calculated p-value $0.03$. Does that imply that the observation has to lie in the critical region? I mean $3\%$ of the distribution is at least as extreme and the critical region is the most extreme 5% of the distribution, therefore $t$ must be contained in the critical region?

  • $\begingroup$ That would be true if the t test is the appropriate test to use. Remember that the t test depends on normality. $\endgroup$ – Michael R. Chernick Mar 5 '17 at 18:16
  • $\begingroup$ what is observed test statistic ? What do I understand from critical region ? $\endgroup$ – Subhash C. Davar May 12 '18 at 4:48
  • 1
    $\begingroup$ Most often the value of the statistic and the p-value are related by a monotonous function. E.g. for a one sided t-test you have $$p=\int_{t_{observed}}^\infty f_\nu(t) dt$$ with $f_\nu(t)$ the t-distribution with $\nu$ degrees of freedom. So a smaller $p$ implies a larger $t_{observed}$. A $t_{observed}$ associated with a 0.03 p-value is in the 'region' of all t values that are larger than the t value associated with a 0.05 p-value. While this is all true Michael refers to the fact that a calculated p-value might not be realistic (underestimated) such that 0.03 may not be a critical value. $\endgroup$ – Sextus Empiricus May 12 '18 at 6:21

If p-value<α does the observed test statistics always belongs to critical region?

Yes, that's right.

(It doesn't depend on whether the t-test is appropriate as suggested in comments -- the appropriateness of the assumptions doesn't come into this at all; this is a question of the decision you make when presented with a p-value. The appropriateness of the assumptions would matter when interpreting the p-value and it would matter in relation to the decision-process yielding the properties you desire, but none of that is at issue.)

I mean 3% of the distribution is at least as extreme and the critical region is the most extreme 5% of the distribution

This is correct. Anything up to (and including) 5% is at least as extreme as 5%.

| cite | improve this answer | |
  • $\begingroup$ what is critical region in the present context ? $\endgroup$ – Subhash C. Davar May 12 '18 at 4:41
  • $\begingroup$ The term carries its usual meaning. It's the set of values of the test statistic that you would reject at a given significance level. For example, for a two-tailed t-test it is the largest values of $|t|$ that are in the critical region - specifically the fraction $\alpha$ (under the null distribution) of the largest values $\endgroup$ – Glen_b May 12 '18 at 6:30
  • $\begingroup$ This description has an orientation that whether t computed is significant or not significant. p-statistics connotes rejection rate of null hypothesis when it is true. If null distribution matches the observed distribution, and p is low, it means that model is fitted well and predicts well. it is not comparing t statistic with critical t- statistics to acertain a significant or nonsignificant effect. p-statistics evaluates goodness of fit - similar to F-statistic. A low rate of rejection of null hypothesis is indicative of a higher level of fitness of observed distribution. $\endgroup$ – Subhash C. Davar May 12 '18 at 9:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.