I am analysing a rare disease, meaning that a low sample size is inevitable (n = 50 for one group in my case). And I came to a question described as follows.

Could we somehow differentiate the following two non-significance scenarios in terms of interpretation?

A - point estimates are quite the same between the groups

B - point estimates are different but CIs overlap, leading to an insignificant p-value

I have scenario A in my data. Can I somehow say that scenario A provides more evidence to support the non-difference between the groups? How to say it in statistical language?

enter image description here

  • $\begingroup$ This is an interesting question (+1) but also seems like a roundabout way of doing equivalence testing like TOST, particularly if you don’t want to declare an effect size of practical equivalence. $\endgroup$
    – Dave
    Jun 26 at 14:09
  • $\begingroup$ This has been extremely well studied so please read more. And never compare separate confidence intervals when trying to judge a difference. Compute the confidence interval for the difference. And don't assume that point estimates are magical. They have quite a fuzz around them. $\endgroup$ Jun 26 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.