4
$\begingroup$

I have performed an unpaired t-test comparing the means of two groups. My hypothesis was that the stroke group mean would be significantly higher than the control group mean. However, the t-test revealed no significant difference. Here are the results:

control mean   stroke mean   p value    95% confidence interval
   0.0864        0.0927       0.76         -0.0388 to 0.0263

I understand that the range of the CI is high, suggesting that the study was underpowered. However, do the CI infer anything else about the possible true effect? So, the CI is located more within the negative zone than the positive zone. Does this mean that it is more likely that the control mean could be higher than the stroke mean?

$\endgroup$
  • $\begingroup$ Maybe I am misunderstanding something, but is not the stroke mean 0.0063 higher (0.0927 - 0.0864)? $\endgroup$ – Mankka Apr 22 '15 at 22:13
7
$\begingroup$

The most useful aspect of a statistical test is that it warns us against overinterpreting our results. Humans are very good at "seeing" paterns in random noise. Think for example of a Rorschach test. A statistical test is there to warn us when that happens. It does so by giving us a non-significant result. So I am afraid your result is just that your study is underpowered.

$\endgroup$
  • 1
    $\begingroup$ It is unusual to have an asymmetric CI from a t-test, have you double-checked the options used to run it and made sure they match your design? If you haven't already, you should also evaluate your data for adhering to the assumptions of the test. In general, the true difference of means is expected to be anywhere within the CI, it is no more likely to be near the center than near the ends. There are many posts here about Null Hypothesis Significant Testing (NHST), worth checking out given your question. $\endgroup$ – svannoy Apr 22 '15 at 13:52
  • 1
    $\begingroup$ @svannoy I don't think it is unusual at all to have an asymmetric CI from a t test. The CI of the difference will be centered on the actual difference between means. It will only be symmetrical when the actual mean difference is zero. $\endgroup$ – Harvey Motulsky Apr 22 '15 at 15:18
  • $\begingroup$ @HarveyMotulsky I based my comment on the common procedure of computing CIs, namely your test statistic +/- critical value * standard error. I know this formula does not hold for some test statistics, such as Cohen's d, but assumed (I know, my bad), it does hold for the difference of means when you are comparing two independent samples with equal variance. I did a quick search to try and find info to the contrary but didn't see anything. If you have a reference handy, that would be great. $\endgroup$ – svannoy Apr 22 '15 at 17:05
  • $\begingroup$ @svannoy The test statistic for a two-sample t-test is simply $\bar{x}_1 - \bar{x}_2$. (The different varieties of the test are to do with how to calculate the std error.) So the confidence interval will usually be offset from 0. $\endgroup$ – Hong Ooi Apr 22 '15 at 18:37
  • 1
    $\begingroup$ My apology for muddying the water @Kim. I see the CI you provided is symmetrical around your difference of means (math error on my part checking the data). I stand by the notion that the CI should be symmetrical, but around the test statistic not zero. $\endgroup$ – svannoy Apr 22 '15 at 19:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.