Timeline for How are the confidence interval of Cramer's V and the Chi square statistic related?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 18, 2022 at 15:34 | comment | added | Henk van der Kolk | Thanks, I now see much better the relationships between the various topics addressed. However, "the CI for Cramer's V isn't too far off the conclusion from chisq.test()." assures me that everything is still fine. I will run some simulations later, see what happens. All these 'corrections' (both for the chi square test and for Cramers V) make the whole idea less transparent. I was hoping it was simpler than this. But I guess it never is. | |
| Aug 18, 2022 at 15:10 | comment | added | Sal Mangiafico | Still, my advice is that I wouldn't try to use a non-signed effect size statistic (like Cramer's V, that can't be negative) in this manner. The confidence intervals are probably more useful if the point estimate isn't too close to 0 or 1. | |
| Aug 18, 2022 at 15:08 | comment | added | Sal Mangiafico |
Given all those variations, the p-value and confidence intervals won't line up on all permutations of those options. I didn't run such simulations, but just playing around, the CI for Cramer's V isn't too far off the conclusion from chisq.test(). As an example, you might compare M = matrix(c(5,11,10,5), nrow=2) and M2 = matrix(c(5,10,10,5), nrow=2) with correct=FALSE, and compare the various CI's for phi() and cramerV().
|
|
| Aug 18, 2022 at 15:04 | comment | added | Sal Mangiafico | @HenkvanderKolk , you might want to run some simulations on randomly selected data to compare the p-value from the hypothesis test and the range in the confidence intervals. Of course, there are options in the chi-square test (like continuity correction). Bootstrapped confidence intervals will vary inherently because they are generated from randomly selected samples, and there are different options for constructing the bootstrapped confidence intervals ("norm", "perc", "bca", and so on). And, as you note, you can or not apply the bias correction to Cramer's V. (con't) | |
| Aug 18, 2022 at 14:30 | comment | added | Henk van der Kolk | Still, it can be zero, so that should be enough. I just want to make sure that CI and Chi-square are giving me the same result (otherwise I am starting to doubt my basic understanding of inferential statistics). | |
| Aug 18, 2022 at 14:01 | comment | added | Sal Mangiafico | I don't know. Personally, I wouldn't rely on confidence intervals for any effect size statistic that can't be negative for hypothesis testing or inference. | |
| Aug 18, 2022 at 13:40 | comment | added | Henk van der Kolk | The documentation also says: "However, if type="norm", the confidence interval may cross zero." This suggests that that version CAN be used in the context of statistical inference. I am not into bootstrapping, so I do not understand the difference between perc and norm when calculating a confidence interval. But in sum: the combination of the a) bias correction and b) using the 'norm' way of calculating the CI are necessary and sufficient for completely aligning the outcomes of the CI for Cramers V and the Chi square test? | |
| Aug 18, 2022 at 13:17 | history | answered | Sal Mangiafico | CC BY-SA 4.0 |