What does it mean to have a high p value but a very very low effect size? What can I conclude? Is it necessary to report the effect size when p=0.998?
4
-
2$\begingroup$ Have you constructed a confidence interval for the effect size? That may be helpful. $\endgroup$– wzbillingsCommented Oct 31, 2023 at 15:05
-
4$\begingroup$ Why the "but" between "high p value" and "very very low effect size"?? Wouldn't a high p-value be expected to go with a low effect size? There's no hint of a contradiction that would suggest "but". $\endgroup$– Glen_bCommented Oct 31, 2023 at 15:13
-
1$\begingroup$ A high $p$-value is telling you the observations are close to those predicted by the null hypothesis, while a low $p$-value would suggest that the observations appear more extreme under that hypothesis. Seeing $99999$ heads when flipping a possibly biased coin $200000$ times ($p=0.9982$) does not guarantee the coin is unbiased, but does suggest that the extent of any bias is likely to be very small. $\endgroup$– HenryCommented Oct 31, 2023 at 15:25
-
$\begingroup$ This page asks (and answers) essentially the same question: stats.stackexchange.com/questions/330713/… $\endgroup$– Harvey MotulskyCommented Oct 31, 2023 at 18:11
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
There is a strong case for reporting effect sizes alongside p-values, as they both represent two different things. The p-value indicates certainty/uncertainty, whereas the effect size tells you about the magnitude of the effect. You can read this article for example: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3444174/
And refer to this post:
You could report both the effect size and p-value, saying that the difference between the two levels of your variable is small and non significant for example.