0
$\begingroup$

My question is about a retrospective medical study in which no significant difference was found between two different treatments. What exactly is meant by "the lack of a significant difference does not imply equivalence because of the undetermined power of statistical tests"?

$\endgroup$
2
$\begingroup$

They are partly right: No evidence of difference does not imply evidence of no difference. However, even if the power is very high (and determined), you wouldn't be able to conclude equivalence from a non-significant difference. Thus, their explanation is not correct. The only scenario where you can conclude equivalence from a non-significant difference is if the power of the study is 100%. This can only happen in infinitely large studies...

$\endgroup$
  • 2
    $\begingroup$ Agree 100%. And we don't have access to the paper. But perhpas worth noting that as a result, when the words equivalence or non-inferiority are used in medical research there is an explicit or implicit equivalence or non-inferiority margin. Without these margins (and the justification for them) statements regarding equivalence are meaningless. $\endgroup$ – charles Dec 16 '13 at 15:43
  • $\begingroup$ I agree with the theoretical correctness of the above. However, there is strict equivalence, i.e, mean = 0, and there is practical equivalence, mean = 0 $\pm$ some amount considered insignificant. The latter type of equivalence can be addressed with a sufficiently powered test. $\endgroup$ – user31668 Dec 16 '13 at 15:52
1
$\begingroup$

Its saying that the researchers didn't calculate the power of the test (i.e., its ability to detect when the null hypothesis is false), so its possible that the test rejected the null hypothesis even though the null hypothesis is not strictly true. E.g., you are testing to see if the mean is 0. The actual mean is 0.1; however, you only have a small sample so you don't have enough power to detect such a small difference. Therefore, you will likely not reject the null hypothesis that the mean is 0 even though it is not actually true. What the authors are saying is that just because they did not reject their null hypothesis, one should not conclude that the null hypothesis is true.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.