How can I compute a log odds ratio for a within-subject design that could be meta-analyzed with log odds ratios from between-subject designs?

Question

Background and Problem

I have a question concerning a meta-analysis combining effects from between- and within-subject designs using log-odds ratios (OR) as the metric of interest. I am familiar with conducting meta-analyses and will be undertaking my calculations in R (using the metafor and lme4 packages). To provide greater context, the studies in question ask research subjects to make a binary decision with respect to a personal preference across one of two conditions. In some cases, each participant is assigned to a single condition (making only a single binary response); in others, each subject takes part in both conditions (making two binary responses). For now, presume I have the raw data in all cases. The issue I face is how best to calculate an OR that is comparable across design and whether I should take the correlation between conditions into account for the within-subject designs.

My Current Approach

I presently use logistic regression to estimate the OR for between-subject designs. The slope represents the OR and the sampling variance can be calculated by squaring the SE of the slope coefficient. Using this approach produces estimates comparable to equations reported in common texts such The Handbook of Research Synthesis and Meta-Analysis, 2nd Edition (p. 243). I then extend this approach to use a multilevel logistic regression model including a random intercept by subject to estimate the OR for within-subject designs while account for the dependency between conditions. The OR and sampling variance are otherwise calculated in the same fashion.

My Questions:

With this in mind, I would like to ask:

1. Is it reasonable to meta-analytically aggregate OR calculated using standard and multilevel logistic regression?
2. Would it be better to use standard logistic regression for both designs (ignoring the correlation between conditions for the within-subject designs)?

Answer 1

I'll focus in my answer purely on the question on how to compute a (log) OR based on a within-subjects design that is comparable to that from a between-subjects design.

Suppose you have a within-subjects design with these data:

                                condition2
                         decision1   decision2   total
condition1   decision1   s           t           a
             decision2   u           v           b
             total       c           d           n

Note that this is the 'paired-subjects' 2x2 table based on n subjects. This table can be rearranged into a 'between-subjects' 2x2 table:

             decision1   decision2   total
condition1   a           b           n
condition2   c           d           n

Then you can compute what is often called the "marginal OR" with the usual equation for computing an odds ratio with:

$$OR = \frac{ad}{bc}$$

And for meta-analytic purposes, we usually work with the log(OR), so just take the log of that. This value is then comparable to that obtained from a between-subjects design.

However, note that the same n subjects are used to compute the cell entries under condition1 and condition2, so the data are not independent. This needs to be taken into consideration when computing the sampling variance of the marginal log odds ratio. Based on Becker and Balagtas (1993) (see also: Elbourne et al., 2002, and Stedman et al., 2011), we can compute (or to be precise: estimate) the sampling variance of the marginal log(OR) with:

$$Var(log[OR]) = \frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{d} - \frac{2\Delta}{n},$$

where

$$\Delta = n^2 \left(\frac{ns - ac}{abcd}\right).$$

(Recall that $s$ is the upper-left cell count from the paired-subjects table.)

References

Becker, M. P., & Balagtas, C. C. (1993). Marginal modeling of binary cross-over data. Biometrics, 49(4), 997-1009.

Elbourne, D. R., Altman, D. G., Higgins, J. P. T., Curtin, F., Worthington, H. V., & Vail, A. (2002). Meta-analyses involving cross-over trials: Methodological issues. International Journal of Epidemiology, 31(1), 140-149.

Stedman, M. R., Curtin, F., Elbourne, D. R., Kesselheim, A. S., & Brookhart, M. A. (2011). Meta-analyses involving cross-over trials: Methodological issues. International Journal of Epidemiology, 40(6), 1732-1734.

Answer 2

You have the individual participant data so you have two choices (a) a one step approach which since you use R can be implemented in lme4 or (b) a two-step approach where you reduce each study to a single summary statistic and then meta-analyse them using, as you suggest metafor. If you do the single step approach you need to specify study and subject as random effects. Each subject in your between subject designs then forms his/her own cluster in the design with a single observation whereas the other clusters from the within designs have two observations per cluster. The one-step design of course lends itself to the inclusion of person level covariates if you have them.