17
$\begingroup$

Could someone please explain why zero being inside or outside the confidence interval tells you whether or not the difference is significant?

$\endgroup$
2
  • 5
    $\begingroup$ It helps to think of 'significant' in this context as shorthand for 'significantly different from zero'. It's only convention to define a parameter such that when it's 0 it represents a null hypothesis of some kind. $\endgroup$ Commented Oct 21, 2014 at 21:19
  • $\begingroup$ This is overly simplistic and not exactly right. – Michael R. Chernick Nov 19, 2019 at 2:25 Could you please explain why it is not exactly right? Thank You. $\endgroup$ Commented Jan 27, 2023 at 0:47

4 Answers 4

24
$\begingroup$

If the confidence interval (with your chosen level of confidence) includes $0$, that implies you think $0$ is a reasonable possibility for the true value of the difference. In general, by 'significant' people usually mean that they no longer believe the null hypothesis ($0$) is a reasonable possibility. Note that if a $95\%$ CI doesn't include $0$, the $p$-value would be $<.05$, which is the conventional cutoff for 'significance'.

$\endgroup$
2
  • $\begingroup$ Could we get a reference to this? I wanted to include in my write up. $\endgroup$
    – Harshad
    Commented Dec 20, 2019 at 11:57
  • 3
    $\begingroup$ @Harshad, I don't know what a reference would be--this is pretty basic. You could just use an introductory stats textbook. $\endgroup$ Commented Dec 20, 2019 at 12:23
7
$\begingroup$

"Having zero in one's confidence interval implies that a treatment effect has no effect." : This is often how such confidence intervals are interpreted, but this is a mistake. A confidence interval that contains zero is not certainty that there is no treatment effect, but that it is uncertain whether there is a treatment effect. Having zero in one's confidence interval implies that a treatment effect could have a positive or negative effect on the outcome of interest. (DataCamp).

$\endgroup$
3
  • $\begingroup$ The OP was not stated in this way. "non-significant difference" is not at all the same as "no difference". $\endgroup$
    – Michael M
    Commented May 26, 2019 at 15:58
  • $\begingroup$ Obviously : "non-significant difference" is not at all the same as "no difference". – $\endgroup$ Commented May 27, 2019 at 21:41
  • $\begingroup$ Having 0 in the confidence interval implies that the treatment is insignificant, NOT ineffective. Your answer is misleading in some way. $\endgroup$ Commented Mar 6, 2023 at 3:25
2
$\begingroup$

In the simple scenarios you're probably considering, it's a logical equivalence: the point defining the null hypothesis is inside a confidence interval with confidence level $\gamma=1-\alpha$ if and only if the observed value of the test statistic is outside the critical region of a test with size $\alpha$.

Consider the case of the usual $Z$ test. You have a random sample $X_1,\dots,X_n$ from a normal distribution with unknown mean $\mu$ and known variance $\sigma_0^2$. Suppose you want to perform a two tailed hypothesis test with null hypothesis $H_0:\mu=\mu_0$ and alternative $H_A:\mu\ne\mu_0$. The test statistic is $Z=(\bar{X}-\mu_0)/(\sigma_0/\sqrt{n})$ and, under the null hypothesis $H_0$, $Z$ has a standard normal distribution. The critical region is $$ \mathscr{C}_\alpha = ( -\infty, -z_{\alpha/2}] \;\cup\; [z_{\alpha/2} ,\infty). $$ On the other hand, a confidence interval for $\mu$, with confidence level $\gamma=1-\alpha$, is given by $$ \left( \bar{x} - z_{\alpha/2} \frac{\sigma_0}{\sqrt{n}} , \bar{x} + z_{\alpha/2} \frac{\sigma_0}{\sqrt{n}} \right). $$ It follows by simple algebra that $$ \mu_0\in \left( \bar{x} - z_{\alpha/2} \frac{\sigma_0}{\sqrt{n}} , \bar{x} + z_{\alpha/2} \frac{\sigma_0}{\sqrt{n}} \right) \;\;\Leftrightarrow\;\; z_{\text{obs}}=(\bar{x}-\mu_0)/(\sigma_0/\sqrt{n})\notin\mathscr{C}_\alpha. $$ The connection with the usual definition of a $p$-value for this problem is also immediate.

$\endgroup$
-1
$\begingroup$

This has to be explained in the context of interpreting the significance of parameters in building let's say linear regression. Null hypothesis of linear regression says that the predictor coefficients are 0, alternate hypothesis is coefficients are non-zero.Confidence interval tells you the actual coefficient value can lie within that range. If that interval includes 0, that means the actual coefficient value can be zero and that means that the predictor has no relationship with the response variable or it is insignificant in terms of its influence on response variable. Hope that explains.

$\endgroup$
1
  • 3
    $\begingroup$ This is overly simplistic and not exactly right. $\endgroup$ Commented Nov 19, 2019 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.