0
$\begingroup$

For a controlled therapeutic trial, is there any value in comparing the p-value of control group pre and post treatment to the p-value comparing the treatment group pre and post treatment?

I'm reading a study in which values for lameness (evaluated on a 1 to 100 numerical scale) were measured pre treatment, then at times T1, T2, and T3.

At no time point was there a significant difference between treatment group and the control group. The researchers then noted that there was a significant difference between values for the the treatment group at time T0 and T3, but there was not a significant difference between times T0 and T3 in the control group.

They conclude that since there was a significant change in the treatment group, but not in the control group, that there is a significant treatment effect.

Is there any merit to this type of comparison?

Thanks, Ben

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

In short, no. This is a controlled trial so comparisons should be to the control group. This is something that some authors will occasionally write, because they have nothing else positive to write, but to quote Gelman (not sure I am getting the quote 100% right): the difference between significant and non-significant is itself not significant. "Claiming signicance" in this way has quite a large type 1 error rate (there is a paper on that).

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for the input and the link. That is exactly the reference I was hoping existed but could not find. $\endgroup$
    – Ben
    Commented Nov 29, 2016 at 21:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.