Assume we carry out a hypothesis test at the 5% significance level. We have an observed test statistics $t$ with calculated p-value $0.03$. Does that imply that the observation has to lie in the critical region? I mean $3\%$ of the distribution is at least as extreme and the critical region is the most extreme 5% of the distribution, therefore $t$ must be contained in the critical region?
$\begingroup$
$\endgroup$
-
$\begingroup$ That would be true if the t test is the appropriate test to use. Remember that the t test depends on normality. $\endgroup$ – Michael R. Chernick Mar 5 '17 at 18:16
-
$\begingroup$ what is observed test statistic ? What do I understand from critical region ? $\endgroup$ – Subhash C. Davar May 12 '18 at 4:48
-
1$\begingroup$ Most often the value of the statistic and the p-value are related by a monotonous function. E.g. for a one sided t-test you have $$p=\int_{t_{observed}}^\infty f_\nu(t) dt$$ with $f_\nu(t)$ the t-distribution with $\nu$ degrees of freedom. So a smaller $p$ implies a larger $t_{observed}$. A $t_{observed}$ associated with a 0.03 p-value is in the 'region' of all t values that are larger than the t value associated with a 0.05 p-value. While this is all true Michael refers to the fact that a calculated p-value might not be realistic (underestimated) such that 0.03 may not be a critical value. $\endgroup$ – Sextus Empiricus May 12 '18 at 6:21
$\begingroup$
$\endgroup$
If p-value<α does the observed test statistics always belongs to critical region?
Yes, that's right.
(It doesn't depend on whether the t-test is appropriate as suggested in comments -- the appropriateness of the assumptions doesn't come into this at all; this is a question of the decision you make when presented with a p-value. The appropriateness of the assumptions would matter when interpreting the p-value and it would matter in relation to the decision-process yielding the properties you desire, but none of that is at issue.)
I mean 3% of the distribution is at least as extreme and the critical region is the most extreme 5% of the distribution
This is correct. Anything up to (and including) 5% is at least as extreme as 5%.
-
$\begingroup$ what is critical region in the present context ? $\endgroup$ – Subhash C. Davar May 12 '18 at 4:41
-
$\begingroup$ The term carries its usual meaning. It's the set of values of the test statistic that you would reject at a given significance level. For example, for a two-tailed t-test it is the largest values of $|t|$ that are in the critical region - specifically the fraction $\alpha$ (under the null distribution) of the largest values $\endgroup$ – Glen_b May 12 '18 at 6:30
-
$\begingroup$ This description has an orientation that whether t computed is significant or not significant. p-statistics connotes rejection rate of null hypothesis when it is true. If null distribution matches the observed distribution, and p is low, it means that model is fitted well and predicts well. it is not comparing t statistic with critical t- statistics to acertain a significant or nonsignificant effect. p-statistics evaluates goodness of fit - similar to F-statistic. A low rate of rejection of null hypothesis is indicative of a higher level of fitness of observed distribution. $\endgroup$ – Subhash C. Davar May 12 '18 at 9:05
