I have a very basic doubt but I am new to inference statistics so please pardon me. My question is when we do hypothesis testing for two samples, we estimate the confidence interval and then if the confidence interval includes 0, we fail to reject the null hypothesis (if I am not wrong?). Now as far as our critical regions are concerned, we fail to reject the null hypothesis if the t-score lies in the non-critical region. So then aren't these two ideas contrasting? What if our confidence interval consists 0 but then our t-score is in the non-critical region. Do we fail to reject the null or do we accept the null hypothesis?
1 Answer
$\begingroup$
$\endgroup$
6
Usually, test and confidence interval will match so that $p\leq \alpha$ corresponds to $1-\alpha$ confidence intervals (CIs) contains null value of the parameter of interest. This is for example usually the case for a single comparison for continuous data.
"Mismatches" between the two things can occur e.g. when you use variance estimates that are only valid under the null hypothesis for the test statistic, but to ensure CI coverage under alternatives you do use a different estimate for constructing the confidence interval (that e.g. occurs for some popular ways of comparing two binomial proportions). In most cases this kind of discrepancy is small. Similarly, things become more complicated when multiple comparison adjustments are in play. In these cases, the rejection of the null hypothesis should be based on the statistical test rather than the CI.
-
$\begingroup$ I was referring to t-test for comparing two samples. Sorry if I was unclear. I guess my question is more related with failing to reject the null hypothesis with the help of confidence intervals. Do we always reject the null if our confidence interval doesn't include 0? And if we do, then what about the fact that our t-score does not lie in the critical regions, even if it does lie in a confidence interval that includes 0? According to t-tests our t-score says that we should fail to reject the null right? So what do we do in such a situation? $\endgroup$– user173461Commented Aug 14, 2017 at 9:24
-
$\begingroup$ Let's say my confidence interval is (-1.45, -1.56) and my t-score lies in the non-critical region. Assuming a t-critical value of 1.96, we should fail to reject the null hypothesis since our t-score lies in the 95% interval. But don't I have to reject the null because my confidence interval does not contain 0? $\endgroup$– user173461Commented Aug 14, 2017 at 9:43
-
$\begingroup$ That shouldn't be possible, as far as I can see. $\endgroup$ Commented Aug 14, 2017 at 11:08
-
$\begingroup$ For a single t-test and a suitable matching CI, there should never a mismatch. So, for that looking at whether the t-statistic for whether the difference is zero is in the critical region for the two-sided 5% significance level, or whether the 95% CI includes zero is simply the same thing. $\endgroup$– BjörnCommented Aug 14, 2017 at 11:26
-
$\begingroup$ So are you guys saying this isn't statistically possible? Both will occur simultaneously or only either one of these occur? $\endgroup$– user173461Commented Aug 14, 2017 at 18:23