1
$\begingroup$

Background

I am performing a meta-analysis of priming studies, where participants receive one of two primes and then produce one of two responses. In these studies, each participant completes multiple trials. The studies are a mix of within-subjects (each participant receives a mix of prime A and prime B) and between-subjects (participants are randomly assigned to receive prime A or prime B) designs.

Studies report the results in the following format:

Condition Response A Response B
Prime A 40 10
Prime B 32 18

More recent studies use logistic regression to analyse results and earlier studies use ANOVA so I have decided to calculate a log odds ratio from the raw data provided for my meta-analysis.

Problem

I am having trouble with calculating the variance of the log odds ratio in this situation. Most examples I can find assume a clinical trial, wherein each data point is an individual patient's outcome. The standard formula is:

$$V=\frac{1}{A} +\frac{1}{B} +\frac{1}{C} +\frac{1}{D}$$

where A, B, C, and D are the number of observations in each cell of the 2 x 2 data. For my example data:

$$V=\frac{1}{40} + \frac{1}{10} +\frac{1}{32} +\frac{1}{18} = 0.212$$

However, if the data comes from a between-subjects design where 10 participants completed 10 trials each, with 5 participants in condition A and 5 in condition B, then I need to control for the dependence caused by multiple observations per participant (within not across conditions). Could I use the formula adjusted as below? Or is there one I am not aware of?

$$V=\frac{1}{p(A)N_A} +\frac{1}{p(B)N_B} +\frac{1}{p(C)N_C} +\frac{1}{p(D)N_D}$$ $$V=\frac{1}{0.8\times5} +\frac{1}{0.2\times5} +\frac{1}{0.64\times5} +\frac{1}{0.36\times5}$$

Finally, as I mentioned earlier, some studies are within-subjects and some are between-subjects. If the data came from 10 participants who completed 10 trials each, 5 trials in condition A and 5 trials in condition B, it would be:

$$V=\frac{1}{0.8\times10} +\frac{1}{0.2\times10} +\frac{1}{0.64\times10} +\frac{1}{0.36\times10} = 1.06$$

However, this doesn't account for the fact that the data in each condition are not independent but came from the same set of participants. I found the formula in the link below, which accounts for this dependence, but I am not sure whether it applies here where there are also multiple trials per participant.

How to calculate LOR variance for within-subjects data

In summary, my questions:

  1. Can I use my suggested formula to control for dependence from multiple observations per participant within conditions in between-subjects designs?
  2. Or is there another more correct formula?
  3. Can I then adjust that formula for within-subjects designs (multiple observations per participant within and across conditions) using the suggested adjustment in the linked question/answer?

I will be performing my analysis in R using metafor, however, any advice on the statistical concepts above is much appreciated, thank you!

$\endgroup$
2
  • $\begingroup$ If they have performed logistic regression you should already have the log odds ratio and its variance. $\endgroup$
    – mdewey
    Feb 2, 2022 at 11:49
  • $\begingroup$ Unfortunately many studies performed ANOVAs not logistic regression. Those that did use logistic regression differ in their random effects structure and in additional variables included in the model so they wouldn't be comparable. $\endgroup$ Feb 3, 2022 at 3:06

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.