In order to improve statstical reliance and science reproducibilty, several authors have proposed to refer to confidence interval fot the difference between two means rather than p-value.
To be able to make statistical inference on confidence interval, some researchers gave visual guides to overlap between confidence interval and infer statistical significance from this overlap.
However, i recently red papersthat doesn't gave the same criterion.
In particular, when testing significance for two independent sample, Andy Field in his book and Cumming(2009) say that :
For a comparison of two independent means, twotailed p = 0.05 when the overlap of the 95 per cent CIs is no more than about half the average margin of error, that is when POL is about 0.5 or less.
However, in another article, written by Pfister & Janczik (2014) they gave this criterion :
Importantly, conclusions based on the CI are valid only for the diﬀerence between the means, and the CI thus corresponds to the t-test for two independent samples. If centered around one of the means this test is signifcant if, and only if, the CI does not include the other mean.
Wich is less restrictive.
What do you think about it? Should we always refer to the more restrictive criterion? Or maybe there is something i miss that makes a differences between those two articles?
Thank you for your answers.
Cumming, G. (2009). Inference by eye: reading the overlap of independent confidence intervals. Statistics in medicine, 28(2), 205-220.
Pfister, R., & Janczyk, M. (2013). Confidence intervals for two sample means: Calculation, interpretation, and a few simple rules. Advances in Cognitive Psychology, 9(2), 74.