Background
I am performing a meta-analysis of priming studies, where participants receive one of two primes and then produce one of two responses. In these studies, each participant completes multiple trials. The studies are a mix of within-subjects (each participant receives a mix of prime A and prime B) and between-subjects (participants are randomly assigned to receive prime A or prime B) designs.
Studies report the results in the following format:
| Condition | Response A | Response B |
|---|---|---|
| Prime A | 40 | 10 |
| Prime B | 32 | 18 |
More recent studies use logistic regression to analyse results and earlier studies use ANOVA so I have decided to calculate a log odds ratio from the raw data provided for my meta-analysis.
Problem
I am having trouble with calculating the variance of the log odds ratio in this situation. Most examples I can find assume a clinical trial, wherein each data point is an individual patient's outcome. The standard formula is:
$$V=\frac{1}{A} +\frac{1}{B} +\frac{1}{C} +\frac{1}{D}$$
where A, B, C, and D are the number of observations in each cell of the 2 x 2 data. For my example data:
$$V=\frac{1}{40} + \frac{1}{10} +\frac{1}{32} +\frac{1}{18} = 0.212$$
However, if the data comes from a between-subjects design where 10 participants completed 10 trials each, with 5 participants in condition A and 5 in condition B, then I need to control for the dependence caused by multiple observations per participant (within not across conditions). Could I use the formula adjusted as below? Or is there one I am not aware of?
$$V=\frac{1}{p(A)N_A} +\frac{1}{p(B)N_B} +\frac{1}{p(C)N_C} +\frac{1}{p(D)N_D}$$ $$V=\frac{1}{0.8\times5} +\frac{1}{0.2\times5} +\frac{1}{0.64\times5} +\frac{1}{0.36\times5}$$
Finally, as I mentioned earlier, some studies are within-subjects and some are between-subjects. If the data came from 10 participants who completed 10 trials each, 5 trials in condition A and 5 trials in condition B, it would be:
$$V=\frac{1}{0.8\times10} +\frac{1}{0.2\times10} +\frac{1}{0.64\times10} +\frac{1}{0.36\times10} = 1.06$$
However, this doesn't account for the fact that the data in each condition are not independent but came from the same set of participants. I found the formula in the link below, which accounts for this dependence, but I am not sure whether it applies here where there are also multiple trials per participant.
How to calculate LOR variance for within-subjects data
In summary, my questions:
- Can I use my suggested formula to control for dependence from multiple observations per participant within conditions in between-subjects designs?
- Or is there another more correct formula?
- Can I then adjust that formula for within-subjects designs (multiple observations per participant within and across conditions) using the suggested adjustment in the linked question/answer?
I will be performing my analysis in R using metafor, however, any advice on the statistical concepts above is much appreciated, thank you!
