# Why is the $p$-value significant but the confidence interval includes zero in a $t$-test?

I am interested in learning whether the two groups below are statistically significant:

ex0112 = (
BP       Diet
1   8    FishOil
2  13    FishOil
3  10    FishOil
4  14    FishOil
5   2    FishOil
6   1    FishOil
7   0    FishOil
8  -6 RegularOil
9   1 RegularOil
10  1 RegularOil
11  2 RegularOil
12 -3 RegularOil
13 -4 RegularOil
14  3 RegularOil


I run the following t-test:

t.test(BP ~ Diet, data = ex0112, conf.level = 0.99)


The results are:

    Welch Two Sample t-test

data:  BP by Diet
t = 3.0109, df = 9.7071, p-value = 0.01352
alternative hypothesis: true difference in means between group FishOil and group RegularOil is not equal to 0
99 percent confidence interval:
-0.4610773 15.8896488
sample estimates:
mean in group FishOil mean in group RegularOil
6.8571429               -0.8571429


As you can see, the p-value for this test is

p-value = 0.014


which is significant at .05, but the confidence interval is

99 percent confidence interval:
-0.4610773 15.8896488


which includes zero, and hence makes the difference between the groups insignificant. How can this be explained?

• Why would you expect that a 99% CI (i.e.e $1-\alpha=0.99; \alpha=0.01$) would correspond to a 5% ($\alpha=0.05$) test?? Oct 17, 2021 at 23:13
• The part I got wrong was that once I set the conf.level = .99, the p value will change to .01 automatically, and if it is not significant, p value would be above it. Obviously I wasn't thinking clearly. Oct 18, 2021 at 1:19

You're calculating a $$99\,\%$$-confidence interval. If you decided on a significance level of $$0.01$$, you would not reject the null hypothesis based on the $$p$$-value, because it's larger than $$0.01$$. If you want to use a significance level of $$0.05$$, corresponding to a $$95\,\%$$-confidence interval, specify conf.level = 0.95:

dat <- data.frame(
BP = c(8, 13, 10, 14, 2, 1, 0, -6, 1, 1, 2, -3, -4, 3)
, Diet = rep(c("FishOil", "RegularOil"), each = 7)
)

t.test(BP~Diet, data = dat, conf.level = 0.95)

data:  BP by Diet
t = 3.0109, df = 9.7071, p-value = 0.01352
alternative hypothesis: true difference in means between group FishOil and group RegularOil is not equal to 0
95 percent confidence interval:
1.98202 13.44655
sample estimates:
mean in group FishOil mean in group RegularOil
6.8571429               -0.8571429


The $$95\,\%$$-confidence interval $$(1.98; 13.45)$$ does not include $$0$$, as expected.