I conducted a paired samples t-test to analyse the effect of an exercise intervention on blood pressure measurements. The test demonstrated a significant p-value. However, when I have calculated 95% confidence intervals for the mean difference between pre and post measures the result suggested there to be no significant difference between pre and post measures, i.e lower level minus and upper level positive. Is this possible?

  • 2
    $\begingroup$ How did you calculate the 95% confidence interval and what is the sample size? $\endgroup$ – user10525 Aug 14 '12 at 13:07
  • $\begingroup$ The sample size is 10 and I calculated the Confidence interval by finding the mean difference between pre and post mean values, then found the square root of the STD of group one squared/ 10 plus the square root of the STD of group two squared/10. Then I multiplied this figure by 1.96. and added or substracted this value from the mean diff to get upper and lower level confidence intervals . $\endgroup$ – Gina Rutherford Aug 14 '12 at 13:23
  • 3
    $\begingroup$ Your intuition is right--this shouldn't happen. However it can happen if two different procedures are used. Eg, 1 thing I've seen several times is someone runs a paired t-test & then presents the results w/ a bar chart w/ 1 bar for pre & 1 for post, & the CI's overlap, causing confusion & anxiety. The analogous plot would have just one bar for differences w/ a CI that doesn't include 0. We need to know a lot more about your data & how you got these discrepant values, as @Procrastinator suggests. $\endgroup$ – gung - Reinstate Monica Aug 14 '12 at 13:23
  • 1
    $\begingroup$ @GinaRutherford Have a look at this tutorial in R. Also, note that t.test(data1,data2,paired=TRUE) in R gives you a confidence interval based on this test. $\endgroup$ – user10525 Aug 14 '12 at 13:40
  • 1
    $\begingroup$ The procedure you used to calculate the CI is incorrect. Use the procedure outlined by @drknexus below in the second paragraph. $\endgroup$ – gung - Reinstate Monica Aug 14 '12 at 13:43

A 95% CI and a test done with an alpha of .05 are completely equivalent. There must be a problem either with your test or your calculation of the CI. As gung suggests, this is more likely a result of how you calculated your CI than a problem with the test.

Based on your descriptions in the comments, you found the right effect size, but your error term is wrong as is your critical value. A paired samples t-test is equivalent to a one-sample t-test of the differences between group (against H0 that the mean difference score is 0). So the error term you want is the standard error of the mean for a one-sample t-test (if memory serves, the STD of the difference scores over the square root of n). You also want to use the critical value for a 9 df t-test, 2.262157, I believe, not 1.96.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.