1
$\begingroup$

I am interested in learning whether the two groups below are statistically significant:

ex0112 = (
   BP       Diet
1   8    FishOil
2  13    FishOil
3  10    FishOil
4  14    FishOil
5   2    FishOil
6   1    FishOil
7   0    FishOil
8  -6 RegularOil
9   1 RegularOil
10  1 RegularOil
11  2 RegularOil
12 -3 RegularOil
13 -4 RegularOil
14  3 RegularOil

I run the following t-test:

t.test(BP ~ Diet, data = ex0112, conf.level = 0.99)

The results are:

    Welch Two Sample t-test

data:  BP by Diet
t = 3.0109, df = 9.7071, p-value = 0.01352
alternative hypothesis: true difference in means between group FishOil and group RegularOil is not equal to 0
99 percent confidence interval:
 -0.4610773 15.8896488
sample estimates:
   mean in group FishOil mean in group RegularOil 
               6.8571429               -0.8571429 

As you can see, the p-value for this test is

p-value = 0.014 

which is significant at .05, but the confidence interval is

99 percent confidence interval:
 -0.4610773 15.8896488

which includes zero, and hence makes the difference between the groups insignificant. How can this be explained?

$\endgroup$
2
  • $\begingroup$ Why would you expect that a 99% CI (i.e.e $1-\alpha=0.99; \alpha=0.01$) would correspond to a 5% ($\alpha=0.05$) test?? $\endgroup$
    – Glen_b
    Oct 17 at 23:13
  • $\begingroup$ The part I got wrong was that once I set the conf.level = .99, the p value will change to .01 automatically, and if it is not significant, p value would be above it. Obviously I wasn't thinking clearly. $\endgroup$
    – Pie-ton
    Oct 18 at 1:19
3
$\begingroup$

You're calculating a $99\,\%$-confidence interval. If you decided on a significance level of $0.01$, you would not reject the null hypothesis based on the $p$-value, because it's larger than $0.01$. If you want to use a significance level of $0.05$, corresponding to a $95\,\%$-confidence interval, specify conf.level = 0.95:

dat <- data.frame(
  BP = c(8, 13, 10, 14, 2, 1, 0, -6, 1, 1, 2, -3, -4, 3)
  , Diet = rep(c("FishOil", "RegularOil"), each = 7)
)

t.test(BP~Diet, data = dat, conf.level = 0.95)

data:  BP by Diet
t = 3.0109, df = 9.7071, p-value = 0.01352
alternative hypothesis: true difference in means between group FishOil and group RegularOil is not equal to 0
95 percent confidence interval:
  1.98202 13.44655
sample estimates:
   mean in group FishOil mean in group RegularOil 
               6.8571429               -0.8571429

The $95\,\%$-confidence interval $(1.98; 13.45)$ does not include $0$, as expected.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.