Rosenberg (2010) discusses the Phi coefficient measure of effect size for $\chi^2$:
$r = \sqrt{\frac{\chi^2}{n}}$,
"where the $\chi^2$ value comes from a two-group contrasts (thus a single degree of freedom), and n is the total number of samples; the sign of the correlation needs to be determined from independent study of the contrast."
Rosenberg criticizes use of the Phi coefficient for meta-analysis on the grounds it “has an underlying, never-stated assumption which is sometimes violated, particularly for genetics studies: it assumes that the expected values from the $\chi^2$ test are equal for both groups.” Instead Rosenberg advocates the general conversion form $r = \sqrt{\frac{\chi^2}{nk}}$. Here $k$ refers to a ratio of expected values for the two groups $k:1$, where $k$ is the larger group. Thus the general formula would reduce to the regular Phi coefficient if $k=1$.
In passing Rosenberg mentions that:
$\chi^2$ tests with more than one degree of freedom are unfocused omnibus tests, and require a much more complicated procedure for conversion to an effect size.
Rosenberg refers readers to Rosenthal and Rosnow (1985), Rosenthal and Rosnow (1991), and Rosenthal, Rosnow, and Rubin (2000) for further details. However, I don’t have access to any of those books, and what meta-commentary (e.g. Boik, 2001) I’ve read has generally been fairly dismissive of the Rosenthal/Rosnow approach, which makes me wonder how helpful the books would be in any case.
My reason for interest in this question is that I'm looking to conduct a meta-analysis in which I aggregate effect sizes from $F$, $t$, and $\chi^2$ tests. The paper Open Science Foundation (2015) proposes a general framework for doing this, as explained in this question. However, in relation to $\chi^2$ tests they propose only the Phi coefficient, which will only work if $df=1$. Therefore I am looking for a correlation-based effect size that will be compatible with the other effect sizes mentioned in that Open Science Foundation (2015) paper.
My questions are:
- Is there a procedure for converting $\chi^2$ tests to an effect size when $df > 1$, that would be compatible with the Phi coefficient and the other correlation-based measures mentioned by Open Science Foundation (2015)?
- Does the procedure also suffer from the problem Rosenberg identified regarding an assumption the expected values from the $\chi^2$ test are equal for all groups? If so, is there any good way to get around that problem?
- Is the procedure limited to particular applications of the $\chi^2$ test, e.g. limited to analyses of contingency tables?
Boik, R. J. (2001). Contrasts and effect sizes in behavioral research: A correlational approach. Journal of the American Statistical Association, 96(456), 1528.
Rosenberg, M. S. (2010). A generalized formula for converting chi-square tests to effect sizes for meta-analysis. PloS one, 5(4), e10059.
Rosenthal, R., & Rosnow, R. L. (1985). Contrast analysis: Focused comparisons in the analysis of variance. CUP Archive.
Rosenthal, R., & Rosnow, R. L. (1991). Essentials of Behavioral Research: Methods and Data Analysis. McGraw-Hill.
Rosenthal, R., Rosnow, R. L., & Rubin, D. B. (2000). Contrasts and effect sizes in behavioral research: A correlational approach. Cambridge University Press.
