1
$\begingroup$

I have taken weight measurements before and after a treatment. When I run a paired sample t-test and if there is a significant change, would it be possible to make the claim "95% of the time treatment increases weight by x units" ?

$\endgroup$
3
$\begingroup$

I have taken weight measurements before and after a treatment. When I run a paired sample t-test and if there is a significant change, would it be possible to make the claim "95% of the time treatment increases weight by x units" ?

Technically no because that's not the correct interpretation of the 95% confidence interval. That 95%CI just indicates that if this study is repeated for many times, approximately 95% of them will have their confidence intervals including the true weight difference (which is unknown to us).

Now, the tricky parts are: i) you really don't know if your study is among those 95%, or the unfortunate 5% which committed a type I error (mistakenly rejecting the null hypothesis when it's true.) ii) 95% of them will include the true difference doesn't mean that exact true difference is what your study found; aka your statement is implying that what your study found is the population's true difference, which is quite unlikely. And iii) the statement is vague enough to be interpreted in the individual level (e.g. 95 out of 100 clients will gain weight or 95% success rate), which is not entirely correct because 95%CI are for population mean inference.

Because of the reasons above, I'd advise against using the proposed interpretation.

$\endgroup$
1
$\begingroup$

I'll state it more firmly than @Penguin_knight. Your interpretation is completely wrong (but a common misinterpretation). The results of a paired t test let you make a conclusions about the average change. It tells you absolutely nothing about possible heterogeneity between subjects.

$\endgroup$
  • $\begingroup$ Thank for the replies. Is it possible to run another type of analysis to come to this conclusion? $\endgroup$ – Lulu Oct 24 '14 at 15:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.